Possessive over my baby?

I have a feeling that once I have my baby that I will not want anyone to hold him besides myself and my husband and my mom. At least for a while. A lot of people say that I will welcome the help but I can't see myself wanting anyone else to touch him. 
Is this silly? Or is there other moms like that out there?