Emotional affair and boundaries

My husband recently had an emotional affair with a coworker. They still work together, and occasionally have lunch at the same time. My husband thinks it is okay for him to go to her house on lunch just the two of them, to smoke(medical marijuana). Opinions?