So how often do you visit your family in law??

So I been married for almost a year now and at the beginning I tough it was okay to go and visit my parent in law every weekend but now I feel that I don't have any time to be with my husband, he likes to go every Saturday that he is off and I'm not feeling comfortable any more. I told him one time that we don't have to go all the time and he kinda got mad about it. My dad and my sister live here in the same state than us but we just go and see them like once a month or so at least that he goes with me. Idk maybe he is a mamas boy or idk what else to do or how I feel about that