Has anyone moved out of state away from family with their husband?

My husband is from central Oregon, and I’m from Southern California. His dad lives in SoCal as well, which is how we met. He came down here in high school to live with his dad for a bit and ended up staying and he’s now a cop here and we’ve been married for 5 years, and are expecting our second son in May. We recently went back to OR to visit family and he now really wants to lateral up to a department up there and move. Not because of his family, but because of the way of life there. SoCal is very hustle and bustle for the money and Oregon is very outdoorsy and he loves the lifestyle. I’m so conflicted. We’ve had this fight back and fourth since we met and got serious. He always wanted to move back and I always wanted to stay because I’m extremely close with my family. I’m starting to consider the idea and I’m wondering if anyone else has been through this? I actually love the state of Oregon. It’s beautiful and I love the outdoors and think it’s a wonderful place to raise kids. I’m just really scared to leave my family. Nothing else holds me in CA except them.