I have dreams of a man I knew years ago

I’ve been with my boyfriend almost three years and I love him very very much and I would never do anything to hurt him or be disloyal. That’s why I feel like such an asshole about this. I know I can’t control my dreams, but it still feels like I’m lying to him or cheating on some weird level. The dreams are never sexual, but they’re always romantic. And they’re always about this guy that I had a thing for almost 5 years ago. We didn’t even know each other more than 2 years. We never even technically dated. He asked me out but I had already moved very far away and I said no bc I wasn’t willing to try a long distance relationship. But at the time I really really liked him and he really liked me too. I even drunk texted him several times after rejecting him. He also has a girlfriend now and it just feels very wrong and creepy to still think about him when we’re both taken and happy. I’m just very confused and I don’t know what my dreams are trying to tell me. I don’t think that I still have feelings for him, but why do I feel so guilty? Has anything like this ever happened to anyone else, and how did you handle it?