whats wrong with me?

been with my boyfriend for 4 years, he's my second serious relationship and the only person I've had any sort of sexual relationship with. I find myself thinking of being with other people too often, like I've considered breaking up.. is it just because he's the only person I've been with? he treats me amazingly, so could it be I only love the way he loves me? I hope I'm not that selfish but I feel terrible