called me his ex's name

So my boyfriend of 5 months (been friends for years) and I decided to take a break in our relationship because we were both slacking really bad in class and getting stressed and it was just causing us to fight a lot. I found out about a week into the break that his ex started texting him and trying to go out with him but he swears they didn't. So anyway, it's been a month since we took that break but for the last 2 weeks we've been staying with each other and talking and just making things better. His ex even called him and tried to get him to go to dinner while she was in town but he told her no and instead made me dinner. So I thought we were doing really well and I just ignored his ex. He went home for the weekend and barely talked to me. came back last night acting like everything was fine. Today after we had sex, we're laying there naked talking and I don't even remember what was said but he started to say her name when he meant to say mine. He instantly started apologizing and making up reasons why he said it but idk. I'm confused on whether or not I should be mad or if there's something more to it. What should I do? I tried talking to him but he just kept saying "baby I'm so sorry I didn't mean to say that. It doesn't mean anything"