How many days do you wait before taking your lo in to the dr

My little guy is congested and has a low fever on and off for the past 3 days of 99. I’m not sure if I should take him in or if that will just be exposing him to more germs. Thoughts....