Unsafe

What do you do when you feel like your kids are safe around certain family members. I had let mykids go with their dad one day to a cookout. And his family memebers we're fist fighting in front of them. Which they do all the time infront of their owns. Well I dont want my kids around them anymore for it. Their dad on the other hand don't agree. We cant seem to come to a understanding about this. What can i do? Am i wrong?