I think my boyfriend thinks I’m over protective

I had a talk with my bf about when our baby boy comes I want only close family around him for the first bit because of germs and stuff. He kinda gives me a weird look when I mention it and says we should have him around people to build his immune system. I get anxiety really badly and just want to protect him for the first bit especially. Is this more a motherly thing to worry about?