When to bring baby around sick family?!

Mo

If I don’t bring my son around sick people, will he not get the immunity he needs? (For reference, he breastfeeds and is vaccinated of course.) A family member keeps telling me to let my son “build up his immune system.” He’s 16 weeks old, at what age can he safely be around someone with a cold?! I keep telling my family I don’t know.. I just know not yet.. Help!