My Boyfriend Still Talks to his exgirlfriend's family?!

I have been dating this guy for three years now and right now I feel hurt and betrayed. I met his female friend *Olivia back when we first dated and I though she was nice at first. I also met her younger sister *Ella shortly after. For years I was friends with both of these girls until I found out Ella used to date my boyfriend and Olivia dated my boyfriend's friend and she once hooked up with my boyfriend.
   What also makes the situation even worse is that my boyfriend is close with their family who he met a year before he dated me. I made it very clear that it's an issue to me. I never had any problems with him having friends of the opposite sex but I have a problem with these girls. He never cheated on me but he hid this from me. 
I don't know what to do and I honestly don't trust him anymore. I also don't trust those girls who I once considered my friends. What should I do? I also have been living with him for two years now. I feel so betrayed and hurt I had enough.