Need some input on boyfriend's family

Ok we've been together for almost 6 years we're expecting our second baby soon an I'm to the point I'm not sure if I want his family involved anymore well some of them we live with his grandma she raised him so he doesn't want to leave an at the moment can't afford our own place but I definitely plan on moving an getting on our feet well his momma in my opinion isn't even his momma she doesn't really have anything to do with him but has 4 other kids that she gets everything for new vehicle new phones all while he gets nothing from her well those other kids are wild disrespectful trashy don't pick up after themselves act like they're better than everyone they're lazy they lie like dogs I'm not the only one that thinks this his grandma has talked about them like this before but I'm to the point where if they come down I'm going to start leaving I don't like them around my daughter an definitely don't want them around the new baby so should I tell his mom if they don't get their shit together that they won't be apart of my kids life of should I not say anything an just leave when they come down my SO doesn't care because he's use to it an they aren't down that often but I can't take it anymore I don't want to start problems with them but I'm so done taking their shit

I mentioned the new vehicle an phones because he's the oldest an his younger brother got a new vehicle when he turned 16 while my boyfriend was 17 with a kid an she didn't think to get him anything he is an adult an he's bought his own truck without her help or anyone else helping get it his grandma has helped him with everything before an after having a kid an his mom hasn't done anything

This whole post was never about her giving us money or anything else that was just more of a back story😒 literally not the point here

We both have our own vehicles that no one else bought for us we bought them ourselves