thoughts on your bf thinking of other girls????

So, my boyfriend recently brought up sexual thoughts and said that he either thinks back on what we've done or thinks of other people he knows. I was listing different girls afterwards and he was like "only some of those are wrong" but he wouldn't tell me who. Later on I asked why he wouldn't tell me and he said "you're acting crazy and if I tell you you'll act even crazier. They're my personal thoughts and I'm allowed to have them." So I just gave up. But it's made me feel kinda insecure because I'm not HALF as attractive of the girls I was listing, but i don't wanna bring it back up because I said I had dropped it but it's been eating at me a lot