In law DRAMA 😥

Okay I SERIOUSLY need some advice.. About a week ago my Fiancé's mother called asked if she could take our 7 month old daughter, who she has only seen twice, 2 hours away for over a week. My fiancé is sadly incarcerated so I 3 wayed the phone for him to talk to her. Out of his own will he told her no because he was not comfortable with it. His mother & family have only seen our daughter twice which was because i drove her down to see them, did not get her anything at the baby shower or Christmas & won't drive here to see her.. They also have not seen my fiancé in a year. So this caused a HUGE argument with them basically ending by telling me I'm refusing to let them see her (which IS NOT TRUE AT ALL) & putting this all into his head (NOT TRUE!). I ended it on the best note I could saying they could see her whenever they would like & if there were any further issues to contact my fiancé. Well this morning I woke up to a FB message from his aunt calling aunt calling me a bitch & if I think I'm going to break up their family I have another thing coming to me, then threatening me with bodily harm. I mean seriously? I'm lost on what to do, I've tried my best to keep a relationship, sending Christmas gifts, pictures, etc but clearly it's all backfired.. What to do??