should I take my toddler or not?

I was invited to a birthday dinner for my brother in law. But I'm hesitant to go and take my toddler (he wants to go) because one of the babies that are going to be there has the croup.. And with me having a newborn baby at home I am scared he will come in contact with it and bring it home to the baby. is this crazy of me or logical to keep him home and not go.?