Domestic abuse?

Does domestic abuse have to be violent? Can there be good days?

I’ve been with my partner 4 years with two children, he’s in charge of all the money and spends it on weed.. calls me stupid ugly fat doesn’t pay any attention to the children. Calls me a bad mum and that the kids will grow up to hate me and that I don’t love them when their my world. If certain things aren’t done the correct way I will get told him like a little child. I’ve tried to kick him out but he always worms his way back in telling me I’ll never do better I’m stupid so no one will ever want me. I work and look after both children he doesn’t work or do anything really. I took a big step and got in contact with a domestic abuse agency they’ve put me on a 1 to 1 waiting list so I can actually have someone to speak to and make a plan either try and get him out me and the children move away. Did I do the right thing? He’s being alright at the moment it’s taken a massive toll on my mental health I start to believe the things he tells me. 😔 it’s not just as easy as kicking him out when he doesn’t leave. We are both 22.