don't know how to go about this.. long but need advice

I just had my first baby almost 2 weeks ago. I'll be home with him until the first week in September then I'll be going back to school full time. My boyfriend will be with him on his days off(which is during the week) then his mom will watch him on the other days. But Saturday's I might have my mom watch him because I know his mom doesn't want to babysit that much, and my mom said she would Saturday's. But thinking about it I don't know how comfortable I am with my mom watching him because of my step dad. I trust my mom 100% and if it was just her I'd let her watch him all the time. But because of my step dad I just don't know. When they first started dating for the longest time we got along really good, he was the best step dad you could get. Then when I turned 17 he just went completely pyscho. I literally did nothing but go to school and come home and stay home all the time. I didn't ever go out at that time, wasn't drinking, wasn't smoking, wasn't doing drugs. And he freaked out all the time that I was "out of control". And he accused me of doing fucking herion! He'd even tell people about it saying shit like "you see the needle marks in her arms?" The only drug I ever touched was pot. And I drank and smoked a lot when I was 15 and I stopped doing it after that. Things got so bad I ended up moving in with my aunt and uncle. He also around that time was pretty abusive to my mom. I've never seen him physically hit her, but I know he has. I've heard it when they'd be fighting, some things she's said before. He was really verbally abusive. After a while of being moved out things changed and he got better to where I know he doesn't hit my mom anymore, and our realtionship got better to where we get along. It's been like 4 years now. But those past things still bother me sometimes. And today I just don't feel comfortable because he's not as bad as he use to be, but he can still really be a fucking asshole to my mom. And he acts like he knows everything, like he annoyed me at the hospital because he would say stuff like he knew what was going on with my son(I don't remember what it was to give an example) so I just know if my moms doing something he's going to get at her like she's doing something wrong and treat her like she's dumb. When he doesn't have any kids nor nieces/nephews to know anything. When she knows what she's doing. He also smokes weed daily. Which whatever do you, but I don't want that around my son especially a newborn. And he does it in the house. My mom just puts a blind eye to it. And the other day they stopped over and he made a comment about my son(I don't really want to repeat it) and I know he didn't mean anything by it because he says it about anything but it was just really inappropriate. I just don't know what to do. Because I know if I talked to my mom about it she'll get upset and defend him. And because for the next year they're taking care of all my bills so I can go to school full time. So I know they'd get more upset about it and idk if they'd threaten to not help if I say something. I just don't know what to do about it, because I trust my mom completely and fine with her watching him. I just don't ever want my step dad to be alone with him(which I know he wouldnt be) and just don't feel comfortable leaving him alone with my mom when my step dad is around. She could always come to our house, but I don't know if they'd catch on as to why I don't want him over there. Or if for some reason she rather be home(she only lives 3 minutes away). We also have 3 dogs which she likes all of them but don't know if she'd want to take care of the baby and deal with the dogs, which they do fine and the living room is blocked off which is where we keep the baby. I just don't know how to go about this. If he wonders why he doesn't get to be around the baby, if he tries coming over here if she did watch him at our house, or if she rather be at her house for some reason.
***i should mention I already started a few classes before I had him and made the down payment 

Glow Resources

Let’s Glow

Glow is here for you on your path to pregnancy

Glow helps you navigate your fertility journey with smart tools, personalized insights, and guidance from medical experts who understand what matters most.

25+ million

Users

4.8 stars

200k+ app ratings

20+

Medical advisors