looking for advice...

My Fiancé and I are having a disagreement about what will happen when we have the baby. I agreed to letting family come and see her after she is born at the hospital, but I would really like and appreciate a few days of alone time for the three of us to bond and for me to have a little bit of recovery time and time to get used to breast feeding. He said I'm being selfish and that his family or anyone else who wants to come visit will be welcomed by him. Do I fight him on this or do I just accept it for what it is and back down. I feel sick thinking about passing my newborn around right after I have her. This is our rainbow baby after two losses. I just want to have her to ourselves for a few days! Is that really too much to ask?!