So sick of family lunch

Twice a month we go at my hubby's family for lunch. That means that the only day off my hubby has we have to share it with his family. When we get home, we are tired and he has to prepare paper work for Monday and I with the baby. I don't want to go tomorrow, I want to say I'm not feeling well. The time my hubby is motivated to go out since the baby has arrived, going to his parents. I'm so sick of it. I'm so sick of his behavior. I'm tired and would love to have at least 2 hours baby free. But he's too chicken to go anywhere with the baby alone because he can't handle the tantrums. Any tips what to say to make it believable that I can't go? If he knows I'm faking it, it'll cause an argument. Especially the part that I've been to my parents 3 times this week.