Am I wrong to feel this way

So when I got pregnant, the child’s father didn’t want to be apart of her life. I am almost 8 months, and he apologized for his behavior and wants to step up and be a good father when she comes. My dad passed away last year and I lost my mom last month, so I do want her to have grandparents in her life. His mom wants to be heavily involved, but she wants my child to spend nights over there. I honestly just don’t feel comfortable with that. I would be paranoid leaving my child somewhere else, and I’m not there. Am I wrong to feel this way?