Long post, sorry but i need advice

I grew up in an emotionally abusive home so sometimes i'm a little dense when it comes to noticing it. I'm conditioned to just tune things out so that nothing escalates. However, a couple that we tried swinging with last night mentioned to me that how my husband treats me isn't right, and now i've gotten to thinking that maybe because he's not "as bad" as my mom was, i may have been tuning out emotional abuse from him. But i just don't know.

Everything was great for the first year or so of our relationship, we had each other and that was really all we needed. But we live on a very low income, so we had talked about me earning money as a cam girl on the side. After a little coaxing, i set it up and got started. About a week later he flipped out and claimed i was cheating, because he had apparently been joking about me being a cam girl. We "worked through it". He still holds that over my head to this day every time we argue, saying that everything i say is bullshit and i'm a liar and a cheater and manipulative. I'm made to feel like every argument is my fault. If i say something about the way he says talks to me, i'm making him feel like a monster. We've been together for 5 years, married for 2 months now. He has been asking and trying to convince me to try swinging for 3 years, and i finally agreed when he found a decent couple close to our age (as opposed to twice our ages) and we did our first full swap last night. He went to their house, and the other guy came to our place. My husband went all the way with her, and the other guy and i only did oral. My husband came home before we were done and immediately started yelling at me, calling me names, belittling me in front of the guy while he grabbed his things and left.

Later that night my husband got a message from the couple telling him that the way he treated me wasn't right. My husband told them that they don't know the full story, and defended his actions. He and i talked and he claimed that i was making it personal, therefore cheating on him again and he has every right to be mad.

And that leaves me with today, we had lunch with his mom, sister and grandma. I said maybe 5 words the whole visit. After getting home he commended me for "not painting him as a bad guy" and said that we'll let this all go and just be happy.

After thinking everything all day i'm starting to realise that he basically said the same thing all those years ago, and now he has another thing to hold over my head.

He's never said anything in front of the kids, but if it escalates i'm not sure how i can avoid it becoming a cycle in my girls' lives. Honestly i just need someone to tell me if i'm overthinking everything, or if there's really something very wrong here.