Baby fever

Hi everyone, my 7 wk baby boy hit the 100.4 F (rectal) mark and we took him to hospital. They checked his urine and blood and everything looked normal. The next few days he gets warm at around sunset to about 100.1 or so.. He's been to the Dr. A couple of times since his ER visit and doctors are puzzled/think he's fine. They think it's a viral infection that he's fighting. I'm on day 6 now.. He eats, sleeps, poops fine.. 
Has any of you experienced low grade temperature with their little ones? I'm extremely anxious and sick worried.