TRUST ISSUES

i have trust issues with him. (we have kids, 5 years together, not married, moving apart for a couple months because of jobs) So i think that he may be bringing someone to the house because he has been trying to come over to me instead of me going over there to him (like usual). I want to drive all the way over there (5hr drive) and see for myself sometime during the time he is off. IS THIS INSANE? or should i just point blank ASK and be Forward with my thoughts? what do u guys think?