My Relationship is Crumbling....

Just need to vent... Recently (in the last 5-6 months) I moved states which is only a five hour drive, but my fiancé moved to be with me shortly after. We were so happy to be together and in our own little home to rent together and all was right with the world. He then started missing family and friends and has visited a few times because they are only a 4 hour drive away but I have noticed we are slowly falling apart. He doesn't want to be here and wants to move back in a few months without me because I am finally happy here and he doesn't want to take that away, but I am crumbling because we are 10 weeks pregnant and all I want is to watch our baby grow together, and to have a happy family but there is no changing his mind. He said he will drive back when I go into labor and asked if I want to make it work, but how can I do that when he is just going to leave me and our baby ? I am so distraught and heartbroken that I don't know what to do ... I feel like instead of being excited for the life that we created I am just counting down the days until he leaves me all alone... And i don't have anyone to talk to about this that won't judge him terribly and all I can think is that I caused this by moving... I don't know what to do...i am breaking and all I can do is put on a front when I am around him...