Orgasm as a form of control?

I’ve been divorced a year now. My ex & I were married 8 years. It was very toxic. Verbally abusive, controlling, self centered….it ended with one known affair, think there were probably others.

Anyway, I’ve dated a few people since. Only one managed to bring me to orgasm, once. I will physically stop them from fingering me, doing oral, or allowing them to bring me to orgasm. It’s almost a panicky feeling if I get close and I’ll think of something else to stop it. It’s almost like I’m not “attached” to them and won’t get hurt if and when it ends if I don’t give them power over something so personal and intimate.

I can make myself orgasm all day. Ex had no problems, it’s just the “new” people. Does that sound crazy or have others experienced this? It’s caused issues with almost all of them.

Edit: I have sex with partners, I just won’t orgasm at all during the act.