Is there ever a perfect time to be with someone?

After nearly 5 years secretly loving each other, we finally confessed about one year ago but then..he moved..to the opposite side of the US..for a career. I would not be the one to stop him from doing so, so I didn’t be like “stay for me”..I have a good job here but being with him would mean so much more. We weren’t together all those years because we were afraid of it not working. We are in our late 20’s, and still have those fears. Lately, I have been having these thoughts. Tell me if they are irrotational or not. I want to leave my comfort zone to give him and I a shot. I haven’t told him that I am willing to move for him. Should I? Or should I keep quiet? Regardless of it may not work. I can see myself marrying and having his children.

Also, what does he mean by “its always a good time?” Why are we so hesitant to go for it?

Advice please