Virgin dating non-virgin... Having jealousy issues…

To those of you who were a virgin when you met your significant other, but your significant other was not a virgin when they met you, how do you deal? I was a virgin when I met my boyfriend, he was not a virgin when he met me, in fact he had slept with five other women prior, one of which he slept with for two years. For the first year and a half of our relationship, I didn't really think about it much, but for the past half year it's been bugging the absolute crap out of me. He and I are sexually active, and I think to myself "how could he have shared something so intimate with five other people??" It really eats at me. It bothers me daily, I think about it every time we're together, and anytime we're getting intimate. Now, it has become such an issue that I don't even enjoy getting intimate anymore. I feel like the only answer is to break up and date a virgin. But then I think, then my future boyfriend will have to deal with a girlfriend who is not a virgin. I feel like I messed up, I feel like I should have waited till marriage… How do I make this stop bothering me?? Or should I just break up?? It's horrible because I love my boyfriend, but this is such a burden.