Advice on his ex's Family.

My boyfriend still talks to his ex's family. He doesn't talk to her (to my knowledge) but he still talks to her family. Before me and him got together, he use to hang out with his ex and her family and would double date with her new boyfriend and his new girlfriend. Is that strange or what? He says that they were able to do that because there was no longer any feelings there. I have the mindset that once you split up with someone, you split up with the family too. He says that her family was always there for him when he needed them. Well, it pisses me off and I'm not comfortable with being around any of them. So, since I'm not comfortable with that, he wants to make plans with her cousin by himself. He tells me that her cousin is very sheltered and treated badly at home by his father and that he looks up to my boyfriend. I understand that, but still. What do you ladies think? I need advice.