Anyone else feel like they have a fragile baby

When it comes down to getting sick?! I get that their immune system is still developing but I feel like every time we leave the house he comes back with a cold.