Why I’m losing my man.

I’ve been dating this guy for almost 5 months. Things have been going great, he’s kind and considerate, romantic and funny, built and good in bed. We talked for two months on FaceTime before we met while we were talking ( before we just met each because of our ex mates). The first night we actually hung out alone, we ended up getting drunk and slept with each other.. he’s 2 hours away so ofc whenever he comes down( which is every other weekend or more) we miss each other physically and emotionally. I’ve decided to stop having sex with him just to see where his true intentions are.. I feel like there’s no way he’s just in it for the sex because of how he acts. He makes the effort and acts like a gentleman should. I guess what I’m trying to forgive out is, it seems like our relationship is built off of that one night of sex. If a couple starts off having sex in a relationship, is it meant to fail.