How would you deal with your partner never willing to answer you if you asked them -"Am I the only person you're currently sexually active with?

I have asked him on several occasions if im the last person he's been with, or when was the last time he anticipated or participated in sexual relations with someone other than me. We began as an open relationship with the complete transparency agreement. Than he asked me to only be with him, and since i already was only with him i said yes. He claimed his love was forever and im very special to him. But i caught him in so many lies & even found women's things in his house he tried to say he thought were mines. And even caught him with a women he was with for over 7yrs at his house right after they had sex the night after my birthday (2021) the same one he was a no call no show on my bday (2020) till 5:55 a.m the day after my bday but i just gave birth to our son 2 months before the incident. He had the nerve to say he didn't cum for her. (Were now going on 5yrs) he still says he will never trust me to ever tell me anything again since i said if he tells me the truth i wont get mad. I got mad but not because what he did with this other person but because he never made my bday up because he said my bday was yesterday so why would we still celebrate it? I told him the only way i could trust him is if he could feel comfortable letting me know if he's been with someone else open & honestly. Without us getting into a heated situation that i end up with no answer and silent treatment. Feeling unwanted & confused. He agreed but i asked him am i the last person you been sexual with? Or am i the only one you are giving your body to? And he says don't ask him nothing. He says believe whatever you want. I say no i can't stay close to you if you rather i leave than just say yes I'm the only one or yes I've been with more than just you. I don't know i feel i need to ask other people since i don't talk to anyone about this. I don't want to talk bad about him to anyone i know. Hope you guys read this and help me if I'm being stupid.