What to do?

So we moved back near our families. We are directly in the middle. My mother in law in insisting on stopping by once a week? Am I being crazy as my husband is hardly home as is and weekends will be my only time with him. I don’t want to sound harsh but I just can’t handle that.