Cant have sex?

So, I'm 22 and a virgin. My boyfriend and I tried to have sex yesterday, but his penis just wouldnt fit. We were both stimulated and we even put lube on the condom, but it was just too painful and it was hard for the both of us to put it in. Any thoughts and advices? Thank you!