am i the problem?

my boyfriend gets hard so easily and he loves when i give him blowjobs and handjobs and stuff but then when we go to have sex he gets soft so we never have sex. he was a virgin before but we have done it once a few months ago and he came and he loved it. but now he gets soft as soon as we try to go from foreplay to having sex. he says its not me and i just make him really nervous but i still feel like its me. theres nothin wrong with my body, im small and skinny and got curves and all that and my pussy is clean so its not like hes repulsed by me.....but like....maybe he is. i dont know what to think, advice?