Am I right to set these boundaries

My boyfriend and I agreed to not let our kid sleepover anyone’s house until they’re potty trained, can speak, and ask to go. There would be no reason as of right now for our kid to sleepover since I’ll be at home taking care and nursing him. My mother has accepted our decision. His mom has not accepted our decision. The part I’m worried about is her freaking cats climbing all over the baby because she wants to leave the door open instead of using the baby monitor. It’s only fair though to keep boundaries the same with everyone.

We have told his mom multiple times, and it’s to the point where I don’t want to be at his parents house to heal after birth. So if I had to I’ll have my brothers and boyfriend carry me up stairs.

Is it wrong to set boundaries with grandparents?