Small family fiancé doesn’t get it

So my fiancé has a huge family that he is close too. For his 22nd bday his parents had a party and he had over 150 people there mainly family. I on the other hand don’t. My mom is an only child and has an aunt and uncle. On my dads side I have 4 aunts/ uncles and 8 cousins none of who I talk to at all. My family has family friends and I have friends but regardless our wedding if mainly going to be his family. His parents have made comments to me before which really hurt since yesI wish I had a huge close family but I don’t my holidays have always consisted of my parents, 2 siblings and grandparents. I know people may not even have this and not complaining at all but when his family rudely makes comments it hurts. My bf recently said in a rude tone so what family of yours are we inviting. I explained how this hurt me and he apologized. But the next day told me his parents said he has to fly home for the holidays. We are engaged like I don’t understand why he can’t celebrate with me and my family. But at the same time he is used to huge parties so probably wouldn’t even be happy celebrating with my family... it just sucks. He told me I can come but I don’t want to leave my parents and siblings plus I have work the day after Christmas is it’s not realistic.