Need to vent and need opinions.

Rebecca • First time single mum to my Penelope Ailsa 💜
I'm moving out of my s/o's parents house, and into an apartment by myself, for me and my daughter only. My boyfriend knows this and is fine with it, although he is a little disappointed. 
However, my mum is really supportive and helpful, and is going to help with supporting me financially and helping once my girl is here. My boyfriend keeps making snidey comments, such as "oh your mum is going to take the baby away from you". And says that he'll never get to see her. 
As far as I'm concerned, he can play as active a role as he likes, I'm not going to stop him seeing his daughter, and my mum doesn't want any more children! She has her own! 
I really don't want my baby to have his surname.. And I don't like any of the names he likes. I want my girl to be called Penelope Ailsa. And to have my surname. 
Our relationship is so hard at the moment, he's a drug user, and I am just so sick of being stressed and anxious. That's why I'm moving out. I just need reassurance that I'm doing the right thing? 😥