Hello ladies

I am in a tricky situation. So I have being seeing this guy for while and our relationship have always being casual, friends with benefits kind of situation. The ending of last year he told me he will be moving out of state for a year for work but will come on the weekend to visit. Fast forward to the present. Couple of weeks ago he start asking me if I will come visit him and then two days ago when I was with him he told me he wants me to come and visit him during the times that he cannot come down. My issue with the whole thing is that we are not in a relationship. And he has never verbal said he wants us to being in relationship or express any interest in one. If anything he have made it a point to avoid things that will define what we have as a relationship. If we were in a relationship I will love to visit him but for me to drive hours to see someone who I share a causal relationship with does not feel right to me. I have avoid or change the topic whenever he try to come up with discussion of me visiting him. Should I talk to him about how I feel about the whole thing and tell him where I stand? Or am I over thinking it?