I had sex for the first time last week and it was... kinda funny

I've been hooked on a guy for years; we dated, broke up and an on-again-off-again mess. I had always imagined him being the one I slept with first, but because of my self conciousness when it some to my size I always backed out. 
My house was empty last weekend (I still with my parents and brother) and invited him over and we hooked up. I just bit the bullet and let it happen. It wasn't bad but certainly not great. I was disappointed that it didn't happen how I pictured it but was happy it was said and done. Here's where it get kinda funny...
We've known each other for 14 years, I've been hooked on him for 8 years. We finally got the ball rolling, and in comfortable place. I'm beyond bliss because I finally get to sleep with him and then he whips it out. He was way, way smaller than I thought thought he was. I'm pretty sure my vibrator was bigger and it's a basic one. We tried different positions and speeds and everything and I just was getting there. Needless to say he finished, again, but I didn't.
All my waiting and hoping had been come to this big flop - which I knew in reality would happen. But I expected like you know awkwardness because I farted or be cause he couldn't get it up or something. I wanna sleep with him again and see if we can make it work but i don't know if I really want to sleep with him again. That sounds petty right?
Someone help me out here. Outside of reminding the first time is never really good, should I continue seeing him if it just isn't doing it for me?