Need advice

My 4 year old daughter within the last month or so doesn't want to go over to my mother in law house she cries and tells me she doesn't want to go , so I told my mother in law that I won't force her to go if she doesn't want to , it started an argument between the 2 of us , then she mentions to me that if my daughter doesn't listen she should be able to spank her I told her no I don't agree with other people besides me and my husband to do that , that started another argument .... my daughter can't really explain herself because she has speech problems ... am I wrong for not making my daughter go over to her grandma house and not allowing her grandma to spank her if need to ... mind u grandma only sees her grandchild maybe 1 time a month