Never dealt with a long distance "thing" before!? Going slightly crazy

So I'll try to keep this short. But basically, I've known this guy for like 6 years (let's call him Jim). we used to work together but when we were friends we both were in relationships, but always got along really well and had a huge attraction towards each other. We both continued to date our SO's and all was well and good, until he and his gf broke up and he moved to LA and I didn't really hear anything about him for the 10 months he lived there. Fast forward to about 4 months ago- my boyfriend and I had also broken up. My friend and I are hanging out and what do you know- my friend sees on Snapchat that Jim is back in town with his brother, who my friend thinks is cute. My heart dropped because I hadn't seen him in soooo long! So we invite them over and had so much fun. Jim and I end up hooking up and it felt like a LONG time coming. We basically talked non stop after that and hungout A LOT for about 3 months straight and every time we hungout was amazing and we just connected on all levels. Unfortunately, this month, Jim had to move back to LA for a job opportunity:( we had a blast the days up until he left, and literally the night before he left he asked me to drive down with him along with his brother and his dad because he wanted me to see LA (and he has family there). I spontaneously agreed, got all my shifts covered. The next day I drove with him 9 hours to LA and spent about 5 days there having a blast and just spending time with him and met his family there. Fast forward until now. I'm home , he's in LA. We talk every day, on the phone sometimes, and constantly tell each other "I miss you" and "wish you were here." The problem is I have no idea how to go about this because one I've never been in a long distance relationship let alone a long distance relationship that isn't even labeled. We both got out of toxic 4 year relationships so I know we should have taken it slow anyway so maybe its good he moved?? To see if we can stand the test of distance AND time?? On top of that my friend took it upon herself to ask him what he thought about me because they are actually friends and this is what he said.

(She's the green). And I don't really see that as a good sign but at the same time I totally get what he's saying because I feel the same too.

My question is I'm supposed to go visit him next month and I want to see him so bad but it's going to make me sad when I'm back home again and still not sure what the fuck we are. Should I just stop talking to him altogether and not get invested?? I'm so confused. On top of everything his family LOVES me and want us to be together. I talk to his dad every day, his grandma who is like his mother figure is like in love with me and I get along with his brothers like they are my siblings. I can't help feeling like if I gave it up I would regret not seeing where it went but it's soooo hard dealing with the distance 😫 any advice/tips/ anything!?