Do I give the money back?

I got 2 Christmas cards from my grandfather who I do not speak to. I don't speak to him for legitimate reasons, he is a bad person and I just try to avoid him. When I had my baby he kept trying to ask about him and see him and I haven't allowed it I don't want him in my sons life because of the type of person he is and past things he has done to my family. He sent us 100 dollars in a Christmas card. Do I send it back or do I just text him and say thanks. I know if I text him he will try to see and ask about my son and I don't want him to be involved in his life at all. But my husband said it's more rude to send the money back. I'm not trying to be rude I just don't want to accept money if I'm not speaking to someone but my husband said I will look rude either way and now idk what to do.....