My hubby can't achieve ejaculation since we stopped using condoms

Since my husband and I stopped using condoms he says having sex feels very different and he can't sustain an erection or he's almost there but he's fatiguing (and quite frankly so am I). It's frustrating to us both of course. We know it sounds silly because usually  men have the opposite problem--not liking the sensation of using a condom.  Any suggestions? Has this happened to anyone?