Afraid family will cause anxiety after baby

Hi yall. I haven't had my baby yet, but I'm worried that all the attention may cause my anxiety to be triggered. I already suffer from anxiety and depression. I know I 'll be a great mom, but I can get overly anxious sometimss. My mom and family can be really overbearing. I'm afraid that directly after the baby is born, they will want to visit everyday and this may cause me anxiety because my safe space will be intruded on (I mean my home). I'm not really sure what to do. I feel like if I tell people to visit only a few times a week or once a week, they may get upset and say that I am being rude about the baby. It's not intentionally to keep the baby away from people, I just don't want to be overstimulated and have a panic attack

Update: These are really great ideas. I have never been a bold person, so I'll have to stand up for myself. Also that visitation calendar idea sounds nice