Is it bad.....

That I’ve never met my husband’s family? He’s originally from Texas. He moved down here(South Carolina ) 16yrs ago to be with his ex. They ended up having kids and he never went back home, even though he wanted to. We met in 2014 and have been together since. We have a kid together and one on the way and I feel horrible about not having met his family. They didn’t come down for our wedding which kind of bothered me. My husband is white & Puerto Rican and dad’s side, which is white, was against him marrying me because I’m black. His mom has always been supportive. Me and his mom talk on the phone and FaceTime from time to time but we’ve never met in person. He’s gone to visit twice since we’ve been together. The first time I was pregnant with our son and didn’t want to get on a plane. The second time he went to his gdad’s funeral and our son was only a couple of months old so we stayed behind. He’s now talking about us moving to Texas which I’m totally against because all of my family is here. He said I’m being selfish because he’s been away from his family for years but I’ve told him that’s not fair because I’m not the reason he moved down here. I completely understand that he’s home sick and wants to be with his family but so do I. It’s not fair for him to ask me to leave mine behind. We’ve planned to visit his family before I get too far along in my pregnancy. He says he just wants me to see where he’s from and maybe that’ll change my mind because I’ve always lived in the country and living in a big city like Dallas will be different. I’m sure I’m not going to change my mind though especially with his family not liking me. I will not bring my kids around that. Granted, he doesn’t really have a rship with his father so he’s promised me that they won’t be an issue. I like my country life but I also don’t want him to hate me for not wanting to go. Idk, am I wrong??

I should add that it’s not his dad who has the issue, it’s his dad’s family.