Help a bi girl out. This is driving me nutters.

So I'm bi just lean a little bit more to the hetero side. Issue is every male relationship I've been in, once they find out its open party on the 3sum question. I've caved to this in the past and wasn't exactly thrilled, wound up ending those relationships.

Now I'm married 5yrs, and a mom. SO knows and recently has been obsessed with the idea of bringing another woman into the relationship or a 3sum. Says he just likes the thrill of talking about it, just kinda un-nerves me a tad because of my past, and the fact that while I was preggers he was flirting online for nudes of his then Facebook friends. So yes there's a pre-existing trust issue.

Anyone else have this issue? Ways to deal please?