Grandparents

I have a 6 year old already but have one on the way with a different man who I've been with for 3 years now. His family is awesome but his mom is talking about how she wants to have days to spend with the baby. My BF is so tired of me bringing this up because I told him, the baby is staying home with me for the first month or two before she is going to go out without me. He doesn't understand the feelings of a mother. We are very overprotective and it's hard to leave the baby early. How do I bring this up when the time comes to the grandma??? I don't want to step on toes but this is MY baby too and not just her sons.  She treats my 6 year old like her own and she's wonderful but I'm not letting baby out right away. I always ignore it when brought up because I hate confrontation. But the time is ticking down and it's going to be brought up when she's here...😩 My BF told me to stop worrying about this. I can't help it!!! I don't want my baby leaving my side. It's causing me anxiety.