Boyfriend problems

Basically I have a caring , loving boyfriend 60% of the time and the other 40% he’s a selfish, manipulative dick and I’m not really sure what to do I know everyone has bad days and is allowed to be rude but he takes it too whole new levels of mean he’ll say I’m a fat ugly pig and considering I’m an athlete I’m none of those things so he seems to think it’s okay because I’m not fat that it doesn’t hurt and it’s funny but it’s not I actually take it too heart sometimes and even when I get upset when he’s said a lot worse he just says I’m being petty and immature and will insist it’s my fault I’m upset not his so he won’t apologize

I don’t know whether I should stay with him because I love him or leave him because he treats me like shite and I’m always last in his priorities because he thinks he’ll never lose me.