Husbands family😡

Okay. So my husband has lived with my family and I since we were 15, when him mom just up and moved one day and he didnt want to go. Were now married and 18. We have already mobed out on our own but pur landlord ended up breaking into our house. We couldnt get into a place right away so we've been living with my parents again until we buy a place, which should be by the end of this year. Well recently we found out we were pregnant! Everyone is very excited, but the only problem is his family. They live over 2 hours away and everything is cheaper up there. We went up for a weekend and the whole time they were trying to convince me to leave my parents and move up there with them, which i could never do bc im too close to my parents and i dont want to take the baby so far away. They wanted nothing to do with us before this baby. They would say nasty things to me and put us down bc were staying with my parents for the time being. I know they want to be close to the baby but i don't know how they can expect me to leave my family. My husband of course would love to move up there but i dont understand bc his mother practically abandoned him and left him with us. Any advice??? Ive already said i dont want to do that and now their trying to bully me into it.