UUUUUP↑ then dowwwwn↓ 😲

So my husband is 40.. I am 32. I've only been with 8 guys my whole life, so I've not came across "ED" previously.. we can be "playing aka foreplay" and he will be hard.. we start to have sex and WHILE we are actually in the act.. he goes limp.. INSIDE ME!?! is it me? (He has not been formally diagnosed with ED but that what I call it) he has never tried any drugs such as viagra or any others.. and this limp incident doesn't happen every time, but like every other time we have sex (at least 2-3 times a day which I love and is amazing) so when he goes limp we take a break and try again later. He ends up getting it to work right eventually. I just feel like it's me, the reason this happens and it breaks my heart. Anyone else experience this?