To date or not to date

Stella
So I have this friend that admitted he liked me an I sorta like him too. We're kinda together but not really. I like talking to him, hanging out, and doing the cute things such as cuddling and kissing. We have had sex 3 times and all I can think about is how small his penis is. I can't even feel it in me. I don't really feel that connection during sex since that's all I can think about. It doesn't help that he's shorter than me either. Point is I'm not sure if I should date him an some advice would really be appreciated.