Leaving partner because of bad fights?

I need some advice. Our baby was born and the first two or three months were great, but then it really went south. We started fighting A LOT. We have so different ideas on how everything should be. My boyfriend has also grown into a whole other person and I guess I have too. Our fights get real ugly. He will call me things, we both yell, slam doors. It's a mess. Our baby is one now and he's underweight and has been since he was born as he was born early. I've read that really bad home situations can affect children's weight and development and I'm so scared this is happening, as he's slightly behind on development milestones too. We're having a really hard time and I really try to make it work. But if this is really the case I'm obviously ready to go through whatever just to make sure my child is healthy. I've tried to make us go to counselling but my boyfriend refuses. I hate our ugly fights. I want to break up every time they happen, but I know that's childish. But I don't know how to deal with it. I can be quiet and nice and do my best to fight good, but he'll still scream and call me names and I can't take that. When we're not fighting everything is great. So it really sucks if we need to break up because of it. He definitely has anger issues and he's aware and apologises thoroughly after every fight, but then the exact same thing happens. I've asked him so many times how hard can it be not to call someone you claim you love ugly names but he'll continue. Are leaving or putting up with it my only options? I don't want my baby growing up calling people names or having bad health because of me.

EDIT: Our child was obviously not planned. I had to be hospitalised my whole pregnancy due to complications so we never had a chance to sit down and discuss anything before our baby was born because we weren't even sure we were going to make it. We used to share the same values, but he has really changed a lot, as I said. We're both 30 and have been together for almost 10 years, so I'd say we have known each other pretty well.