Meeting a man via cheating would it work?

If you meet a man, who cheated on his women to be with you, then left her to be with you, could it ever work? Or would you then always have trust issues knowing he can cheat!