Baby daddy and SO

How do you handle your baby daddy’s SO spending time with your baby? He’s not even born yet and this already gives me extreme anxiety.