should I not let my mother take her anymore?

My mom takes my daughter every other week or so for 2 days. I let her because she wanted to. But she's mad that im pregnant again. She literally has not said ONE word to me about it. But others tell me the things she says. She has told people that she doesnt care about this baby and when they are born, she will just take my daughter and someone else can have the other baby. 
She's in love with my daughter but im not going to do the favorite thing. My husband suggested telling her that if she cant love both of them, she cant have one. Not to mention, we live an hour away. She drives an hour here and back to get her, and then to bring her home. But yet she never stays for a minute to talk. It's always just getting her stuff in the car, taking the baby and going. She doesnt call, doesnt text. Its like im not even here to her, only my daughter. She talks to my other sisters everyday though. 
What do you think? 
Should I stop letting her have my daughter for a while?
Also, just another thing that bothered me, she knew we were struggling, may lose our house and be homeless and not once did she ask how we were or anything. For a few weeks we were living off of ramen noodles (literally) because we had to pay bills. She knew this and just laughed and left. The same thing happened to my sisters a while back and she did everything ahe could, even told them to move in! Im not mad that she didnt give us money because we are doing good now, just mad that she didnt care. My father is always making sure we are okay and shows concern when we arent. It just sucks!