need help

I have been dating this guy for 6 months. In the beginning we had intercourse a couple times a month. Then it went to once a month and now it has been a month. We spend every weekend together and some days out of the week. When I say something joking about something sexual he calls me a perv and tells me that's all I think about. But it's like he does it joking around I think. I know he is not with anyone else. But it makes me think that I am doing something wrong. Please help.