HELP!!!

So there's this guy and we've been a "thing" since November of 2015 when we started talking. We act and do things that couples would do but we're not dating. He doesn't want a relationship because he's leaving next May for the Army (No he's not just playing me, he talks to literally no one besides me and his best friend, he hates everyone else at our school lol) so anyways, in March we talked about having sex, and at first it made me kind of upset cause he didn't wanna date me but he wanted to have sex with me. I'm over that now cause honestly I don't even want a relationship at this point either. So in May I went over to his house and we tried to have sex but I was too scared and got as far as taking my shirt off before I told him I couldn't.
We're not as close as we were back in May so he hasn't brought it back up. But now I'm ready and I don't know how to tell him. When I was leaving his house I asked him if he ever wanted to try again and he said no that he was "too young" which doesn't make sense because he was the one that brought it up in the first place. I really want him to be my first and I'm ready, I just don't want to come off as too strong or crazy, what should I do?