What to do

I've been married to my husband will be two years this Sept. We quit having sex like once in 3months, top it off there been things showing up at my house like a pair of pants that are not mine, a pair of underwear , make up. All in which he has no idea who they belong to. Night before last I washed my two young childrens clothes who are 9 and 6 and mine and my husband's clothes. I put them in the dryer. When I was taking the clothes out the dryer I found a condom in the dryer. I asked about it he got real defensive then took off in the car. when he got back home he accused it being my son's and then tired to say it's mine. Funny thing a out this is i am highly allergic to latex, and what in God's name would my 9 yr old do with it or how he got it? Seriously now he still sticking it's mine. I'm the whore. I told him I'm done I can't do it anymore and now he throwing up if I loved him I wouldn't end it I would work through it and he swears on everything condom ain't his. I'm no fool. I did do the right thing right? I really need help to figure this out.