Condom Issue? (Long Post Sorry)

Va

Me and my boyfriend just started dating not too long ago and we ended up sleeping together a few times (all times using a condom of course). I personally didn't expect it to happen so soon cause I'm extremely paranoid about getting pregnant and I was going to wait until after I got my IUD, but shit happens in the heat of the moment. I'm currently not on birth control but I am getting my IUD tomorrow so that issue is fixed.

Regardless, when I'm using birth control (was previously on the pill), I refused to have sex without condoms with my ex because I wanted as little chance as possible to get pregnant. That in the past was fine because we were each other's first so he wasn't used to unprotected sex since sex begun with me.

The guy I'm dating now has had a few previous partners whom he was used to having condomless sex with. So the two times we've slept together, he almost immediately went soft once the condom was on which made things super difficult. Few strokes in and then it just wasn't working. I switched things to oral and made him finish that way, but he was having a super hard time staying hard with the condoms. He told me he just has to adapt to using them but I personally don't see that happening since he's not used to the lack of skin on skin stimulation. I guess I'm just afraid that we're never going to have great sex because even after my IUD, I want to use condoms still. It doesn't help that he's a bit smaller than what I was used to for several years so that adds on to things. On the emotional side of things our sex is wonderful, but on the pleasure side it's not doing too well.

I'm sorry that this was super long and rambly. I guess I just feel a bit sad and guilty and I'm not sure exactly what to do.