Friends with Benefits? Help

Me and my Trainer at my gym (not personal trainer, i do some group classes that he teaches) are friends with benefits, but only became that because apparently he can't date clients. We admitted we liked each other more then friends and kissed and went out on a double date before we decided we couldn't because of his job.

Recently I lost my virginity to him and I'm stuck in a place because i really like this guy. We are so similar and really match each other so much, he's met my mum and sister and I've met his sister, his family think I'm just a friend, where as my family knows i like him a lot.

He said he wants to quit his work, but its been a long time since we had the conversation where i ask "if you leave does that mean we can get together" I want to ask him, but I don't want him to say "we are only friends with benefits and we won't go anywhere" I could be overreacting but I really like this guy and I want more. Do i just ask him the question?? what if he says no and then that ruins of friendship?? Someone help me :(