I keep fantasizing about a threesome when I'm having sex with my fiancé..

I've had a couple threesomes (MFM) in my past. Sometimes if I'm having trouble getting off, I fantasize that I'm back in that situation when I'm having sex with my fiancé. 
Is this wrong? Am I losing my spark with my fiancé??