Advice - infidelity

So I work with an individual that is married and his wife just gave birth to a baby In April. I found out about a month ago thru a friend at work that he has been having an affair with another woman (who is also married) at work. They apparently have sex on their lunch break/ in bathroom etc. but I swore I would not say a word to anyone because she promised the woman that has been doing this that she wouldn't tell anyone but she obviously felt the need to get it off her chest by telling me. I am absolutely disgusted by all of this. In shock never would have believed it. But I need advice. Apparently this dick bag even came to work to have sex with her while he was off for two weeks with his wife and newborn baby!!! I'm sick and disturbed. It has been bothering me.   What would you do in my situation??! I know the right thing to do is inform the significant others but then I'm ruining families and breaking a promise etc !?!! Should I just let it go?