Frustrated with visitors

I know our family members are thrilled about our LO, but I’m over visitors and my house feeling like it has a revolving door. My husband and I both work, and I am teacher so I frequently have to work on school things over the weekend and in the evenings during the week. The constant visitors is so frustrating and annoying especially when you want to relax, spend time with your LO, and get daily/weekly chores done. My MIL made a comment today at lunch and in text tonight about how she would love to come play with our LO even though we met them for lunch. She got mad at my husband when he told her to stop stimulating our LO because he needed a nap (in his car seat while we ate). So I am thinking that might be the reason for the comment, but it’s still annoying that she wants to come see him again so she can “play with him,” even though she saw him today. When we tell my husband’s side of the family it’s not a good time for visitors they get very offended and feel as if they are entitled to see our LO whenever they want, not when it’s best for us or our LO. I EP as well so we don’t like visitors when I am pumping. My husband has basically said I shouldn’t have to pump in an uncomfortable place to appease other people so he tells them they can leave so I can pump comfortably (my MIL has a very hard time with this). I know everyone loves our LO and wants to see him, but this mama is tired and she needs her space. Is anyone else over the visitors??