I need advice

So I'm in a long term live in relationship. I love my boyfriend to death but I'm not sure if I trust him. He was in a ten year relationship and was engaged before me but they broke up four years ago. They still talk all the time. I've met her on multiple occasions. She calls my boyfriend at least once a month and has asked him to hang out. My boyfriend knows how much their relationship bothers me but says they will always be friends. What should I do?