The EX factor

I've been seeing this guy and we've becoming pretty inseparable. I'd like to think we're getting serious as we've met each other's families and been on countless dates and lots of sleepovers. There's just one thing: his "first love" and him broke up 3 months ago and she has a new boyfriend but I swear every chance he gets he brings her up, even if it's irrelevant to what he's talking about. Two nights ago we went to dinner w his family and he was late picking me up because he said he was in a bad mood and wanted to calm down. When I asked him what was wrong that night he said he got in a fight with his mom. Tonight he told me that it was because he had seen his ex and her current boyfriend, he then continued to tell me after our date with his family he went home and cried. I feel like a rebound. I've had a "first love" too and I know what it feels like to see them with another but I would never broadcast that to my partner. And if I were to see him and his girlfriend now (which I do since we live in a small town) it doesn't phase me. I just don't know what to do or how to bring this up. I'm not good with expressing my emotions and irritations in part thanks to my ex and in part by habit. I need help beyond the "tell him it bothers you" aspect.