Sexually curious

So I've only had one partner, my husband. I was 18 he was 24. He has had a crazy sex life ( from all the stories he has told me) and I can't help but feel like I wish I had more experience. When we broke up before we got married I met a guy who I kinda started to like but I'm the midst I got back together with my now husband and married and now we raise our son together. He was the only guy I was ever really sexually attracted to. If I was going to ever have sex with anyone beside the man I lost my virginity to it would have been him. Now married I find myself thinking about it a lot!!! I'm horrible:(. But my question is to ladies that have had multiple partners is it worth it. Like isn't sex just sex or is it completely different with diff people. Am I missing out on something?