How is everyone going to handle visitors at the hospital?

I'm a FTM. It will be just me and my husband for the birth. I would have allowed and wanted my mom and she feels strongly that it should just be us, and I'm fine with that too.

But what about visitors after. I'm not trying to not have visitors but feel like it should be on my terms. Obviously my/our family comes first as far as visiting. But I've had some non family members say "I'm coming to the hospital to see the baby" as if I have no choice.

Given that everything goes according to plan, we will only be in the hospital 24-36 hours after baby arrives. I just don't want it to be a revolving door of visitors to the point where we have no time to rest, adjust to the new baby, learn to breastfeed, etc.

I don't want to hurt anyone's feelings either.

But my goodness, why can't people just say I'll come see him after you get settled in at home?

I just don't feel like it should be a stressful thing, and it already feels stressful. Like I shouldn't have to entertain people. And I don't want the baby passed around constantly.

I'd like some other opinions.