Cheating on my fiance with an ex who's married....

I know we are terrible people. Try and keep the negative comments to a minimum. We are ALL grown and we all know what we are risking and also that it's only gonna end bad. My Question is whats the best way to end it and STAY GONE. we've tried but he then popped up at my job with flowers. He said he'd never do it again. But im still worried. How would you end things???