I need advice.

I've been with my boyfriend for a good two years. He did stupid things that I never held against him in the beginning. He got drunk with girls and flirted with them to "help his brother out" and then he spent time talking to his ex and i walked up on them. I got over it, i focused on the future. You know? But a year ago my ex came up to me sparking conversation, and I was so dumb to even give him a sec of my time. He tried asking me how I'd been and trying to conversate really.  He kissed me out of nowhere. I cried my way home honestly. I never hid it from my bf. I told him right away. I told him I understand if he'd leave. He said he forgave me and we moved forward. We went a year of continuing this relationship. Last night, he told me he never forgave me and he doesn't know what to do know. Now, I don't know what I should do. I never help his shit against him even when I could. I focused on a future with him, I never tried drowning myself in our past. Does anyone have advice on what should be done or what I should do? I'm seeing him later today.