My husband is so anti social?

I am very close to my family. He is not close to his at all so he doesn't understand why I like to be . On his days and time off he just wants it to be him and I and no one else. Lately I have let him win and haven't spent much time with my family at all. But today is my nephews birthday and tomorrow my family wants to go to church, then lunch, then hang out together. My husband is so upset! He just wants this weekend to be about us. BUT IT HAS BEEN EVERY WEEKEND FOR THE PAST MONTH. It's not like I want to spend all day at my nephews birthday or all day with my family tomorrow. Things come up and I would like to go to some of my families get togethers?