Racism in the family !?

Okay, so my younger brother has a daughter and she's almost a year old, and I'm pregnant now and I'm due the beginning of next year . I just recently had my anatomy scan and found out the sex of the baby & announced it to the family as well. My brother lives in another state with his child and gf so we FaceTimed him and let him know that way. My mom was talking to my brother and telling him how we can't wait for them to meet and play with each other, and he says, "My baby isn't playing with that dark baby!"... Yes, I was shocked, hurt and just saddened by that. We all were in disbelief, and I told him that racism starts with the parents & that's not nice or fair to me and my baby. At this point, I don't even know if I want him around my baby because that's just disrespectful to me. I think I'm just looking for different points of views and/or opinions on it. For reference, we're black, he and his gf just have a lighter skin tone, and my man and I are are a darker skin tone. Also, sorry if it's in the wrong group.