Is it just sex or is it more?

I started a new job 4 months ago, from the moment I walked thru those doors my SP ask my best friend (who was already working there) if I was single and to put in a good word for him. when we first started talking (which was just a few days after I started) I told him that I didn't want everyone in the warehouse to know bc I was the new girl and you know how that warehouse gossip and rumors are, he agreed and told me that he wanted that too bc he just got a divorce and his ex was one of those crazy jealous type and still wanted to be with him and used his 2 young children against him when she didn't get her way. So we have been together for 4 months now, but he didn't keep it a secret which is ok bc as far as I know there hadn't been any rumors which is awesome. but we still hadn't officially called our situation a relationship, we talk to each other every day, he calls me babe/baby, we are exclusive, he bought me a gift when he went on his family vacation, his parents know about me and mine know about him (though we never met each other's family) but his mother does ask about me and why I didn't go on the vacation with them. I get jealous of other girls and he gets jealous of other guys. and he says that he does have feelings for me but if I ask about being more than just SB he brings up the situation with his ex and how she is holding his against him (and she is, he hasn't got to see them in over a month not even for Christmas and they don't have any type of custody agreement, and he wants to hire a lawyer but can't until he has the money saved up.) idk what to do. I can feel myself falling for him hard, we can't hardly keep our hands to ourselves all day at work. I need opinions on what this is. so I stay strong and hang in there til he can get through the mess with his ex or do I cut my losses and leave him be? and in case anyone is wondering we are both in our early 30's