Need advice!!!

I don't know what to do... I'm pregnant and I have a toddler and every other day my husband wants to go see his parents after work and on the weekends. At one point his family didn't like me but now I'm having a baby and they are nicer and more accepting of me. But every other day we have to go have dinner at his parents and stay four to five hours. And on the weekends we stay half a day out of the weekend. I know they are old and I always say Yes let's go but I'm getting sick of it!!!!!!
I miss my family time and being with my husband and the kids and cooking and just relaxing as a family. His mom is 72 and the dad is in his 80's. Am I being mean or how do people handle this????