Toddler high fever for last 4 days

3 days ago my 1 year old spiked a high fever in the middle of the night and he’s had it ever since unless he’s on Tylenol or Motrin. He has no other symptoms but 2 nights ago we went to the ER because we don’t have a ped where we just moved. he threw up all over me in the waiting room and the doctors saw no sign of infection anywhere so they assumed gastroenteritis She said just rotate between tylenol and Motrin until the fever goes away and if it’s not the flu then maybe he’ll develop more symptoms to tell us what’s causing it. She said to wait 5 days and see if it’s still high but I’m too freaked out to wait. It’s just non stop high fever of 104 without the meds. He’s so fussy and easily irritated at certain times of day but happy otherwise. He only threw up once in the er and never again. Has anyone’s toddler spiked a long lasting high fever for no reason? Should I call my old ped to get advice?

UPDATE: just would like to add that it was a children’s hospital so she was a pediatrician