Would it be wrong?

To make this story brief, I do not get along with my FIL. He’s racist, narcissistic, and super selfish. He’s talked to my husband about divorcing me & marrying this girl he knows (the hubs shut that convo down quickly), and has continuously made remarks about my family’s race. He’s also wished us both dead & has started rumors that I’m harassing his girlfriend online. Basically, I dislike him A LOT. My husband has recently blocked his number and all social media bc of all of this. We just found out we’re expecting our first, but the thought of our baby having contact with my FIL makes me feel anxious, especially because they’ll be half-Hispanic. I want to talk to my husband about it but I don’t know how to go about starting that conversation. What would you do? Is it selfish of me to not want him in our baby’s life to begin with? I’m so emotional rn and don’t want to make a rash decision.

Edit: I want to thank you all for your words of advice! I refrained from replying to comments to keep anonymity but wanted you guys to know you didn’t go unnoticed! Thank you again :)