His boys dont listen to me and he doesn't help

I try to discipline the kids (4 of them) and when he's home his run to him. And if I try to when he is here he says something or he won't back me up. Like I'll tell the kids not to do something and he'll say (where he thinks they can't hear him) "I don't want to mom" or something similar. I've asked him not to but he just says they can't hear them but I think they can sometimes. I told the youngest to get off of a stool we have and the youngest didn't and my bf just stood by him for like a minute and walked away and left him up there. I'm getting frustrated taking care of 4 kids and them not listening. My 2 sometimes are the same but I try to tell them to listen to him but I dont feel like he helps me with the kids. Everyone always comments on how could I can handle 4 kids but I'm naturally a calm person but I feel like I'm starting to reach my limit (for lack of a better word) I've told him how I feel and he just told be not to give up on us. But nothing has changed. I just need help with these kids, too much for one person sometimes. 3 boys under 6 and 1 girl close to preteen. None see there other parent but three are enrolled in school. I feel bad for feeling like this too. We've been together for 2 years.