Is it wrong not to want any extra visitors at hospital? FTM dont know wat to do

Im 22 and ftm...So our plan was we are basically allowing immediate family at hospital when she is born, our parent's and siblings, is (5 people each). We really want to make this time special and personal with our baby and I don't like having bunch of people and strangers around when I'm exhausted and trying to figure out how to care for baby. I'm already scared, and i wanted to make my time at hospital being able to bond and preparing best for when get home with baby alone and don't have help and can't do that with people constantly coming and visiting. We are only there for like a day, and i want to make the most of it for my baby's benefit. We were going to have a 'visting' party like a week after. (That's what i think is best for baby and us) but his mom is saying that some people are getting upset and butt hurt that we dont want them at hospital (lke it was about them, which i can understand people want to see baby, but what about us wanting it to be special and most benefit. It's not like we are not going to not let them see her.

Idk what to do