Getting it up

I recently tired to have sex with someone for the first time and we been flirting for months and even hinted at doing the deed. So we get down to it and he isn’t hard and It’s weird his been hard around me on other occasions and he continued to say I need to get him hard and that he’s not like other niggas I slept with cause he get pussy which rubbed me the wrong way. So I gave home head and he would hard of and on and he even put it inside me and we fucked for a bit and then it came out and he immediately went soft but I wet and never dealt with before he continued to say it was me which is completely killing my pride rn. Can someone relate or help understand what’s wrong? Is it him or me?