Boyfriend has problems staying hard during sex?

Kimberly
So I'm 18, and my boyfriend is 20; we've been dating for approximately 3 months. At the beginning of our relationship we had sex extremely often (like, close to 3-6 times a day) and we still do. But recently in the past month he's been having problems staying hard. He says it's nothing to do with me. If we go without a condom then there's not a problem, but after he puts one on he usually looses his erection but when we try again later (with a condom still) he's able to stay hard? I've never had a problem with him loosing an erection during oral or even a hand job. I'm wondering if it's maybe the condoms being too small for him because they're a little tighter than he's used too. He also "complains" (for lack of a better word) about how tight I am when he first tries to stick it in and then proceeds to go soft? Sorry this is so long, I wouldn't be asking if it didn't upset him so much to the point where he leaves the room in anger after it happens. Advice please?