Is it normal to wonder what things would be like with another man?

I want to start off by saying that I'm head over heels for the man I'm with now. He isn't perfect and nor am I but he is good to me and he makes me happy. So we've been ttc for over a year and nothing. Everyone around me is getting pregnant including my exs girlfriend. This particular ex (Kareem) and I dated over ten years ago but stayed friends. He is the ONLY man who has truly loved me unconditionally with zero expectation of anything in return. He always went above and beyond for me and was in love with me since I was a teenager. I wasn't into him that way because I was dumb and wanted the guy who didn't want me and ended up getting hurt. He we are years later and almost into our 30s and I just wonder how my life would've been if I had given him a real shot. Is it normal to think like this? I feel so guilty at just the thought.