Visitors Rant

So I'm due in 3 weeks exactly Oct. 2. My grandparents are coming to visit the end of Oct. At first I was reluctant because I wanted my baby to be at least a month old before anyone came to visit. My husband and I are both from SC but we're stationed in TX. So my grandparents are also coming from SC. Well today they said that my aunt and uncle want to come also. I said no. Why? I will still be recovering and don't want to feel overwhelmed and obligated to entertain extra people. I also don't want the baby to be around too many people before he receive his shots. I am not a person who likes a lot of people around so it's bothering me that I think they are still going to come anyway. I am only comfortable with it being my husband, baby, grandparents & I. I don't need extra people especially at that time. Am I rude? I'm going to keep saying no every time they bring up the subject of my aunt and uncle.