HELP! ftm 8 week old baby with upper respiratory bug!

Hey girls! So my sweet baby boy has been sick since Wednesday with congestion and a cough. No fever. We went to the doctor on Thursday and tested negative for RSV. Doc said he's completely healthy, just upper respiratory bug. I thought he was getting better but now today he doesn't want to eat and is sleeping all day. I have to wake him up for feedings so he can stay hydrated. The congestion almost seems better (we do nose frida, humidifier, and steamy showers). But I took his temp rectally and it was 100.3. Im kind of freaking out. Should I take him into the ER or just monitor temp and keep trying to get fluids in him? He is taking about 2 oz every hour or 2. Thanks on advance.