Dealing with fiancés family

I’m not sure if I want anyone from my new fiancé’s family to visit at the hospital. They’ve never been involved in my life nor shown interest in me or the pregnancy. My fiancé has a son that is 10 and is a severe ADHD child with a number of issues. I don’t even want him to visit the new baby and I until I’m comfortable let alone in the same hospital room. I actually have been having second thoughts about being with him due to all the issues that his son has created or caused. What do I do? I’m so uncomfortable with his family. I’m heartbroken because he does not understand.