should I feel bad?

My boyfriend/baby father's mom had a stroke so she can only move and work one arm. She also walks with a cane and needs a lot of help with everyday things. Is it wrong of me that I don't want her holding our now 2 month old? I've been like that since she was born but don't say anything. I'm just sooo scared she will drop her because she is already unsteady and only has one arm. She walks with her and tries to do normal things but can't because of her only one working arm. I feel my baby is going to wiggle out of her arm and she can't catch her because she's unsteady and can't use the other arm. My baby wiggles ALOT. It stresses me out so bad when she comes over, which is ALOT! I just don't say anything because I don't want to hurt her or my boyfriends feelings. I don't even know how I would say or bring that up to her nicely. I guess this is just a rant because now that I'm back at work and I leave my daughter with my boyfriend she ALWAYS goes over there and she's there now and I'm just at work worrying hoping my little girl is ok.