Do I have the right to get mad or not

So my best friend just to me tht her brother in law works with my ex. They had a conversation bout me bout how I'm a freak. Do I have the right to get all mad and defensive or should I let it go. Like I understand guys will always talk bout there ex's I guess I'm just shocked cuz I don't think he would talk bout me negatively. Should I confront him and say something or no