Advice please

Our baby just had his 4 months shots and has been really sick with it. It’s breaking our hearts. My in laws planned us to all go camping on their rv for a week soon but I said if he isn’t feeling better by then not only are we going back to the doctor but I’m going to stay home with him. My in laws love holding and playing with the baby and will try and insist we still go. (Little back story: they can kind of be possessive over him and whenever we’re together they want him the entire time and get grumpy when they can’t hold him). They will say something along the lines of “We can take care of him and give you a break” but I don’t want to. My baby boy needs to just stay home where he’s comfortable and have mommy snuggles until he gets better. How do you suggest I respond to them saying that without hurting their feelings?