Fiancé & Family

My fiancé told me that his family will always come before me no matter what. I just thought in marriage, your spouse always comes first... He says it all the time and it makes me upset sometimes. He said if his brother came over and wanted him to hang out with him all day, he would pick his brother over me. And if I didn't like someone in his family, he would have to leave me because he's had to deal with that in past relationships and his family will always come first. This is really the only thing in our relationship that makes me wonder if we should actually get married or not.. He's perfect & so sweet but when he says family comes before me it hurts :/ What do you think about the whole spouse/family situation?