Fiance caught cheating

I was looking for some items in my duffle bag but my fiancée and I have the same bag. In his bag I found some fetish type toys, a whip, strap on, some other type of whip, and an eye mask that you wear when trying to keep the son out. The box was open in the bag except the strap on was still in the open box and the other items were loose. Anyhow, the mask looked like it had been worn bc it had eyeshadow shimmer on the inside of the mask. He came into the closet where everything was and said, you found my stash with smiles and laughed. But when I asked him who he had been sleeping with he got serious. Got down to his knees to believe him that he wasn't doing anything. When I showed him the eyeshadow he said he nvr used any of the items and said it probably came like that. He denies everything but I just can't believe him. There's no other explanation right? We have a son together which makes it even worse. Then when he realized I wasn't believing him, he got angry and basically kicked himself out and said I would regret my decision. I'm not even in tears bc I feel I have only one choice but to leave. Anyone else think different?

We've been together for 5 years, and the package was still in the bag and it was post marked in September of last year. Either he's been cheating on me since or he did a long time ago. I just can't see it any other way.

Update: He still denies it. Said he bought it for us but didn't know how to introduce to me. Says the mask must've came that way but he knows I'm wanting out but it's hard knowing we have a son together. He's trying really hard and even scheduled counseling for us to get thru this. I don't even know anymore.