No visitors after the baby

It is wrong of me to not allow visitors after I deliver my baby? I mean I don't want anyone there except my baby's father and me. I mean I'm a very private person and a lot of my family understands that. I even ask my parents how they would feel if I didn't allow anyone, which I'm very much like them, they said it whatever makes me feel comfortable, but my baby's father's family may not be so understanding. I feel like be at the hospital I'm there to heal after birth and spend my baby's first moment with the father and me.