Am I being selfish?

 I don't want to move to where my significant other lives because we would have zero support except from his friends family, who I only met once. I keep being told I'm over reacting and that they're my family too. But they aren't. My s/o has no idea the struggles and stress a new born brings onto people. I'm not comfortable with leaving all my support behind just to move to be with him and be even more stressed out than I already am.  I feel like it would be such a bad idea and my s/o is stubborn and it's his way or no way. 
I keep telling myself it's just him panicking but the other part of me thinks he won't budge and I'm going to have to do this alone.