I have a right to be upset

My father in-law just called my mum to say he had someone call him and say i was having an affair. He wont say who told him or who he heard it was with. I have called my husband he is response was "its fine dont be upset its no big deal" Am i right to be upset and hurt i feel so betrayed by his dad as hr believes it cause I've been going to the gym and getting fit.