fiance and I got into an argument

Ok so we're making dinner and start arguing about the stupidest thing. we were watching Austin Powers 2. And well at the end it got us talking and arguing about if one of us went to the past or future and slept with the other would it still be considered cheating. I said no cuz it would still be him.... But he says yes cuz it'd be past or future him.. not present him.

What y'all think? btw it was more of a contemplative talk..