My boyfriends ex is having his baby

So my boyfriend and i have been on and off for 2 years and 9 mo ago we split he slept with his ex baby mama she lied and said she was on birth control and now just gave birth. I told him i was fine with him being there for the birth but now what? Shes obviously not going to let the baby out of her sight and i dont want those two just chilling together in her house. I know an option is just to remove myself from the equation but has anyone been in a similar situation? What did you do? This woman is awful and throws shit in my face all the time.