Should I let people hold my baby during cold season?

First time mom here! I've been stressing about this for the last couple of months. I love my family and my fiancé's family but some of them already told me that they can't wait to hold this kid come thanksgiving but I'm due a few days before and will probably be wanting to rest and the amount of people is making me anxious. What if, by chance I feel up to being out of the house, and I go to this gathering and someone is sick and wants to hold the baby? What if they all do? They're will be so many people. What if an adult has the baby and decides to let a child hold my newborn? Children aren't the most clean or supportive. I don't know if I'm stressing myself out over nothing. I just know that I don't really feel comfortable with a large crowd being touchy with a newborn who may potentially get sick. I feel so pressured to go. I'm also afraid offending people if I want to wait until I feel comfortable enough to have people hold him. What do I do?