Is it wrong?

Is it wrong that I feel uncomfortable because my boyfriend isn't a virgin and I am? He doesn't pressure me and says he'll wait as long as I want to because he says um worth it but it still makes me uncomfortable to think about. Almost to the point that I almost said no when he asked me out but he confessed before he even asked me out. Why do I feel this way?