Baby daddy drama!!

Whittney
My babies father and I have been together 2 years. He is older than I am (43&26) anyways from the get go I didn't trust him. He has 4 kids with 3 different women, very secretive man, never shows his phone to anyone, stays in touch with all his ex's but I gave him a chance. He's sweet and very easy on the eyes, very smooth talker and so forth. After about a year, we broke up and I got pregnant by another man. Before you think oh lord this girl is a whore, it wasn't like that!  I fell for this man, and I made a mistake. I have cried my eyes out, ask God for forgiveness and had to rebuild my trust back from my baby daddy. He went with me to the abortion clinic, was by my side, even said we would get married if I went through with the abortion. I believed him but only to figure out he was going to throw up all in my face. Since then which was almost a year ago, I have gotten pregnant again and now having a baby girl with my first bf. He was very happy from the get go but for about a month now, he has stopped having sex with me. Says he's stressed, depressed, doesn't feel good, and a lot on his mind. It has gotten to me. I have gained a lot of weight, he doesn't seem interested in me, won't build a realationship with my kids, barely comes over, anyways I said something to him about the sex yesterday. I told him I feel neglected and I felt it was either he wasn't attracted to me or was having sex with someone else. He flat out told me it was both. I lost it. I am at a complete loss of words. I ask who and he said he couldn't tell me bc it would start drama. But part of me thinks, I deserve all of this. Every single bit. But the other part of me thinks, how could he do this to me?!? I need advice