Confused

I found out the other night this guy I have been hooking up with for a couple months now is a virgin. I told him I wanted to have sex with him the other night and he wanted to too, but it was too late and I couldn’t sneak out so we ended up not. I saw him again last night, thinking he would wanna hook up. Well apparently he is scared of having sex (all of his friends told me), and he thinks that that is all I want because I am not a virgin. How do I tell him that I don’t care at all about sex? Keep in mind he is 15(almost 16) and I am 17 and ready to be in a relationship. I know he isn’t ready so is it worth my time to wait for him because I know he has some feelings for me, or should I just move on? Sorry this is all over the place I just don’t know what to do anymore he’s the only person I want.