Crazy past.. please don't judge need advice.

I have had three pregnancies I have two girls from my marriage that after five year he was not faith full ending in divorce. Then started dating a old friend from high school we got pg and I had a abortion years ago now. That relationship end again for selfish reasons. Once again bad choice, it still rips me to pieces. Know I'm in a good relationship with my new boyfriend and like all relationships we have a battle a head of us. Together we have three kids his son and my two girls. I get to care for his son and the girls while he works. And I have always be up front with my past it makes me who I am. He has hide lots of stuff drugs and some alcohol and we are work on that. For me its very hard and he brings up my abortion sometime and it hurts him, then makes me feel like I do. Do i deserve to be happy or whatever because I made a bad chioce... I want a life with this man. But so today we where going to bed and he gets so upset like he does and push me away not physically but he makes me feel like I do for my choice I made years ago. I remind him we all make mistakes and even though it was my choice I aloud for my ex to help make that choice. I m a dam good mom I love my kids even the ones I didn't birth (my boyfriend son) since I can't change it I have to live with that everyday. Suck knowing the person I give my heart too hate it as much as I do. Any advice?