Any advise?

So my boyfriend and I have been together for over 3 years. I love him more than my life and we plan on being married once we both have time to save up for our future. He's also my first-ever boyfriend and I know he'll be my last. He has taken all my firsts: first kiss, first sexual experience, V-card, you name it. He has had a few other very short relationships before me and I knew that I wasn't his first girlfriend or anything, however I was under the impression that I was his first sexual experience as well.

We were talking about this kind of in the heat of the moment like 'I'm all yours no one else have ever had me' kind of thing. One thing leads to another and he basically says "well you didn't get the majority of my firsts but you got the most important and I'm so happy it was you" and I basically said "what are you talking about?" Turns out his one girlfriend way back when had given him a few BJ's and after 3 years if being with him I had absolutely no idea. He swears he remembers telling me and felt awful when he saw how shocked I was. That is obviously something I would've remembered hearing. Its been a few days and I can't really seem to get it off my mind. I'm not mad really because it was in the past long before he met me but I'm just unsure of how to really deal with this because it keeps creeping up in the back of my head and I don't like it. I love my boyfriend he would never do anything to hurt me and as he said I did take 'his most important first'(meaning sex) but I still feel almost robbed and I'm not sure how to move myself past it... Any suggestions? Or any similar experiences and how you felt?