Boyfriend hanging out with ex!

Lizzy
My boyfriend is friends with his ex. They are all in the same large friend group but he does go to her house (with his friends and hers) to hangout more often than not. I've explained to him my worries and all he says it "we're friends and that's all we're going to be." Should I be worried?