Sex with someone you are not official with

Well ladies I'm sort of undecided. I come here because I need an outsider's opinion on this. I have been going on dates with a guy that lives miles from me. We don't see each other constant and we talk weekly. We have known each other for about 2 years and there are times we could lose touch at months at a time. Recently when we saw each other I felt more of a connection. He is insinuating sex in our conversations but I feel like I shouldn't yet. We haven't really talked recently about "what are we" & I'm scared of sleeping with him and 1. Leave or 2. Just be "friends". 
A part of me wants to have sex with him but another part of me is scared that that's going to be it. I do care about him, although he says he cares about me too I don't think he does as much as I do or it's all a lie.
Is it a bad idea to have sex with him if he isn't officially your boyfriend (nor have had any talk of it recently) or should I find a way to postpone it and talk to him? 
Please keep your judgments to yourselves, we are both adults but as someone that has been hurt I'm just looking for someone else's opinion. Thank you :)