Really annoyed with other people- please read

Last night I went to meet my brothers girlfriend and it was a nightmare. As I am talking my step father chimes in and tells me not to talk I’ll about other people. I was trying to warn the girl about my sisters behaviour, before she decided to make an appearance. Anyways this man talks badly about everyone and treats everyone like crap. He’s been verbally abusing my mom for 20 plus years and says rude things to my kids. Anyways I think I had had enough last night. I told him to shut up and that he needs to stop living with my mother and just leave. She’s telling me to ignore this behaviour. Was I wrong by not ignoring this. This has gone on for so long now and frankly I am sick of putting up with this. I hope I didn’t scare my brothers girlfriend. As she is a very sweet person. I can’t go back to that house if that man is there. Anyways I have decided to have Christmas, thanksgiving and Easter at my house this year and distance myself away from that man and his rude comments and abuse. What would you do? My mom wants to continue to let this happen but I don’t want to be a part of this nonsense anymore.