Should I even be with him?

I have been friends with benefits with this guy for almost a year. He is four years older than me. I have done so much for him to the point where my friends believed he is taking advantage of my kindness. He has said he wants to take us more seriously. The thing that bothers me is that although I do see him more often we don’t just hang out like I thought we would . All we do is have sex and talk for a little and then he leaves. I asked him what he wanted for Christmas and he said “nothing since I can’t get you anything”.( He has no job) I told him I was getting him something for his bday and he said “no because I already do too much for him”. I am so confused because I feel like he is not taking “us” seriously and I feel like he is taking advantage of me I of course had that conversation with him before where he obviously said he wasn’t. Idk if I should leave him and cut him off entirely since I know I can’t be his friend or not. Any advice?