Sexually attracted to another man besides my boyfriend

Okay so I had this thing with a guy before my current boyfriend. I really liked him. The past guy screwed me over pretty bad and honestly I don't think I ever got over that. I'm extremely in love and SUPER sexually attracted to my boyfriend so don't get me wrong, there is nothing wrong with him. But this guy... I don't want a relationship with him, but I'm so attracted to him sexually. And I have already had a sexual relationship with this guy.  Is this a bad thing?! I'm so scared and I hate myself for thinking this way because I absolutely adore my boyfriend. Should I feel bad for being attracted to another guy?