Not letting baby see grandparents

To make a long story short, I grew up in a very dysfunctional household. My dad used to beat my mom when he drank too much, money was tight, my mom was/is a hoarder and doesn’t clean, left me with a babysitter who molested me, smoked around me and would ash in my eye or burn me when she would smoke while doing my hair. Her and my dad split 20 years ago but are now back together and my dad is still an alcoholic but not violent, he also struggles with mental health issues.

I’ve told my mom if she doesn’t quit smoking she won’t be able to meet or see the baby and I made that very clear years before I got pregnant. She tried for about 12 hours and now has given up(I haven’t nagged her at all)

Basically I don’t want my baby around her toxic behaviour. Being pregnant has brought up a lot of unresolved anger towards her (I’ve done therapy myself but never with her, and tbh I don’t see family therapy happening).

I just don’t know how to actually set those boundaries without being a total b*tch. Anyone else ever been in a similar situation?