Sick baby, at what point should I freak out?!

I have a 3 month old (just shy, 3 months on March 2) and I was just diagnosed with bronchitis and an upper respiratory infection two days ago...so dad has been doing most of the caretaking the last couple days. Baby has two older siblings 8 and 12 in school so I’m worried about germs they may have carried. I’m pretty anal about hand washing, etc but I know I can’t protect him from everything! I’ve noticed today all he’s done is sleep and has hardly wanted to eat, when he’s been awake he’s super fussy and just wants to go right back to sleep. I can’t even keep him awake now, he doesn’t want to play and is just falling asleep no matter what your trying to do with him...and he just has that look in his eyes, you know when you can tell someone isn’t feeling well?! He doesn’t have a fever but I’ve been checking it every 20 min (probably too much but I’m freaking out a little)...should I take him and have him checked out or just wait it out? I’ve never had a baby get sick this young on me😩 to top it off my best friend in the world baby shower for her first baby is tomorrow which I’m helping host so I would absolutely hate to miss it! Ugh what should I do?

Update** I ended up taking him to ER when his temperature reached 101.1...unfortunately he and his dad were both diagnosed with flu A...they said his lungs looked perfect and he seemed well hydrated so they let us go home. He seems to be feeling better today, he hasn’t had a fever since the morning after leaving the hospital and just wants to sleep...today he’s been up playing with my daughter some and being less cranky than he has been lately! Dad has definitely been a bigger baby through all of this than the actual baby lol thank you ladies for everyone who reached out

His grandma lovin’ in him❤️