stopping father from going to the scan?

So me and my fiancé have recently split up because of how he has treated me through my pregnancy (down playing my pregnancy symptoms, telling me he's talking to other people while we was together, going on nights out until early hours of the morning and leaving me home crying) I've then caught him smoking weed recently in his flat (I'm very against weed and my baby's pram was in the flat smelling of weed) am I wrong stopping him from going to my 20 week scan? I wouldn't stop him from seeing his child when he's born but at the moment him being there just stresses me out as last time he came he just ignored me and I cried the whole way home.
Honest answers please I'm in need of advise
Thankyou 

Glow Resources

Let’s Glow

Glow is here for you on your path to pregnancy

Glow helps you navigate your fertility journey with smart tools, personalized insights, and guidance from medical experts who understand what matters most.

25+ million

Users

4.8 stars

200k+ app ratings

20+

Medical advisors