Baby’s father new gf question

My (still married) ex husband broke up with me in May to date his ex and I am currently 21 weeks and I’m not comfortable with letting her around my son. She made jokes about me having a miscarriage to recently giving him baby name ideas and that doesn’t sit right with me. He says it’s on me then if I don’t let him around her so he can’t see his son. Am I a monster for not wanting her around him for a while?