Family controversy over child's race

I am with a wonderful man. When we first started dating, my family (not my parents) threw a fit about it because he's black. Honestly, I was completely surprised by it because they don't go around spouting off racist remarks. I was disowned, they started telling people I was dead so the wouldn't have to talk about us. Eventually they came around and got over it. Now, he and I are having a baby. All of the crap has started again. They can't give me an actual reason for it, they just believe that races shouldn't mix. They acknowledge my white son but are pretending I'm not pregnant and saying that this child won't exist. Should I give them the chance to come back around or just cut my loses?