Jealousy that I shouldn’t have... DUMB RANT

Me and my boyfriend have been together for a year and he took my virginity which I’m glad I waited for the right guy and I love him more than anything but I am unrightfully jealous that I was not his first. I was a one night stand when he was 20 and drunk. And another girl about 2 years ago. I know I don’t have anything to be jealous about or worried in the slightest but a little voice in the back of my head says what if. What if he thinks about them when we’re having sex, what if he’s lonely and dumb when I’m out of town, what if he’s going out with his friends and gets too drunk? I KNOW he wouldn’t do anything to hurt me or risk our relationship but that doesn’t mean someone else wouldn’t take the opportunity. I know one of the girls names and they both go to the university next to our town. He knows I worry about it, he knows I don’t like him going out drinking but I trust him (for the most part) I feel awful for thinking this but I can’t get it out of my head. One of those girls with come back and want him. I just have to trust him more and continue to build a stronger relationship with him.