Is this wrong of me?

Is it wrong of me to tell my boss I won’t stay over at work? A coworker has the flu and is out, so I work tomorrow and they’ve told another girl to ask if I would stay the extra hours, but I got a sinus infection and feel terrible, I’m still going in for my shift (even though I feel terrible) but really don’t want to stay over while I feel bad, I’m worried they’ll be mad. Lately, I’ve said no a lot to staying over due to school, assignments, appointments.