Hospital arrangements for my toddler?

Ashley • Pregnant with #2 Psychologist with a love for wrestling, boxing, and being fit! ❤️❤️❤️ Family is LIFE
So I've been thinking about who I want to be at the hospital when I give birth and I honestly just want my SO there. We have a 4 year old though and I'm not sure if the whole birthing process is appropriate for his age. (Lol) the reason I only want my SO there is because with my first born, it seems like everyone and their cousins were there (not really just both sets of grandparents and our siblings) and wanting to hold my baby and talk to me and all and visit. Not anything wrong with it I know they're just excited! But I was SO tired and just wanted to hold my baby and close my eyes and have dad by our side. 
I also want my 4 year old to be there pretty soon after but I don't want anyone else there until I have my energy. I don't want to rob my family of the experience, but I also don't want to be stressed out this time. I possibly don't even really want anyone there at all. Maybe like come visit when we are settled at home? Idk I'm stuck here.