Am I giving up to easy?

So I have been with my Fiancé since last June, he moved into my this past March and we just had our baby girl in Sept.

Since the end part of my pregnancy our relationship has been hot and cold. One minute we are so happy and the next its hell! I feel like I can deal with a lot but towards the end of the pregnancy my fiancé frustration during arguments has turned into aggression ( a whole lot of yelling, slamming doors, one time he charged at me and another time he threw his phone in my direction towards a wall). When we aren’t arguing it’s like things are PERFECT, he’s respectful kind gentle patient… all of the things I wanted and needed in a relationship. But if I’m being honest it’s more bad than good lately. When we get into it bad, 90% of the time I’m like okay I’m done take your ring back but he won’t go. He calms down, we talk and then we reset. But I’m so tired of all of the drama and the bad days. I always promised myself I would not deal with disrespect again (I left an abusive marriage).

I’m ready to be done! I know I will miss him and the good he brings but I can’t have this craziness around my children and I don’t think I deserve what he’s dishing either (I’m not perfect) but the drama is just too much. Am I quitting too soon? Should love be enough to work through this stuff ?