Relationship struggle

Felicia • 🤘🏻
I'm not sure if I'm happy in my relationship anymore and I don't know if it is worth it to try to improve it or if it would be better to just end it. 
My boyfriend has good qualities, great qualities in fact and we have a lot of fun together - or we used too anyway. But he's done a lot more bad, than good. And over these past few weeks, I've really been taking that realization into consideration. 
For starters, he has trust issues. He always has - right from the get go. Almost every girl he's dated, has cheated on him. And I sympathized for him. But frankly, I can see why, now. He's invaded my personal belongings such as my journals, my purse, my cell phone. He's controlling and hypocritical - I've given up smoking for him, I've given up Snapchat, and guy friends for him - while he's ditched me to go to girls parties, snapchats girls, etc. He makes jokes or comments implying that I cheat on him and send guys inappropriate messages or pictures that I shouldn't be sending. And he has an overall personality that is really hard to deal with sometimes - almost bipolar tendencies. He's very short and irritable towards me. He's punched a hole in my wall. He has thrown things around my apartment. And worst of all - he is not one to apologize. Ever. (Unless I force it out of him - which in that case, it hardly counts)
And I know you all are probably like okay, all of that but apologies are the deal breaker? Whaaaat? 
But hear me out - I am a VERY forgiving person (obviously lol) but I'm forgiving because apologies mean so much to me. I know for some people it's really hard to swallow their pride and admit that they're wrong. So when someone apologizes to me it's like they are saying "Look, I was wrong for doing this and that. I know I hurt you in the process. And you are more important to me than winning an argument or being right." And that is really admirable. So "I'm sorry" actually says a lot. 
My boyfriend is the kind of person that tries to move past things skipping the apology part - by making jokes and trying to get a conversation or a laugh out of me. And that's a bigger slap in the face than the initial issue at hand whether it be big or small - because that kind of attitude comes off as like "whatever I did to make you mad or whatever I did to hurt you, get over and let's pretend like it never happened". Very frustrating. 
Our sex life has taken a weird turn too. It's not like we've been together terribly long that our sex life has become boring. It's kind of like sex is a chore for him. He definitely hasn't lost his drive because he always initiates in sexual activities but it's usually me catering to his needs and it's rarely reciprocated. Which isn't like him...at all so I don't know what has changed. 
I love him. Sometimes I really don't know why lol But I do. But I got out of a very toxic 4 year relationship months prior to meeting my boyfriend - and the last thing I wanted was to end up back in another unhealthy relationship. Any advice would be helpful, really. What are some things I could do to work towards bettering some of these issues? Do you suggest I talk to him? Do you think it's not even worth it to try? 
I'm just really torn. While my heart is telling if you love him, give this relationship everything you've got but on the other hand - my brain is telling me I deserve better.