Sex help!!!!

Ok so I got really high with a guy and we decided to fuck. He said I made him almost cum and every time he stuck his penis in me he would go limp and it would fall out of me  Is it me or him. I'm very inexperienced could that be it or is it because he couldn't stay hard. Sorry if this is too TMI and I am 20 almost 21  and sexually becoming sexually active.