Trust him or dump him?

So on Tuesday my boyfriend and I had a petty disagreement over nothing. He was calling me some pretty nasty names. He said I was retarded, childish, annoying, ugly, that I act high maintenance and all those sorts of names. He knows I have very low self-esteem and don’t love myself. He was also saying if I continue like that it’s gonna be quiet and he’s leave me. I am honestly none of those things. The problem is I have recently been acting different to protect my feelings. A week ago he had a four hour conversation with his ex-girlfriend so I began to distance myself so I wouldn’t be a rebound and I feel as if any girl would be concerned if their significant other was to do so. After he said this I slightly changed my tone of voice but I didn’t retaliate or anything. I just went a bit quiet because the things he were saying sounded like they were coming from his heart and that he meant it. In that moment he called me sook and said I sook over everything when I honestly don’t. He’s the one always doing stupid things like calling his ex for hours, ignoring my calls, hanging up the phone on me and just taking advantage of me emotionally. So the same day he was calling me all those names immediately he’s like “baby girl” so I said “Don’t call me that I obviously don’t mean anything to you, we just talk on the phone”. He didn’t like the fact that I said that. But instead of trying to make the girl he likes feel better and cheer me up after making me sad he says “bet, then let me go call this girl that’s tryna go chill with me since me and you aren’t together” then he hang up the phone on me and didn’t call me back. Apart of me was hoping he’d call me back and say “baby I’m joking” but he didn’t. In the morning I woke up to no miss calls so I stupidly called him and it went straight to voicemail. I called him throughout the entire day and it went straight to voicemail. Then yesterday night when he finally called me he told me the girl came to his house at like 12am-1am and told him to come outside. He said he didn’t give him his address but no one in his state knows his address because he just moved there and one of his HOMEGIRLS probably gave it to her????? So he doesn’t know how this “random” girl got his address and he says he didn’t talk to her before hand, she just randomly dm’d him that’s “she’s here”. He claims he hasn’t had sex since we started being a thing but it’s hard to believe. What should I do? P.s. I can’t leave him. He’s controlling, manipulative, a giant, in love with his ex girlfriend and she’s also in love with him 🙂