BIG MISTAKE

I just started working at this country club a month ago. Anyway one of my coworkers asked if I wanted to go to her friends house and drink with her but she didn't tell me who the friend was. The friend turned out to be my manager and after I got drunk I had sex with him 😳. It meant nothing to me but now he's texting me and then I found out he's messing with my coworker that I went there with and she has a boyfriend and HE has a girlfriend. I told him that it happened because I was drunk and that I'm not interested in any guys right now and that he has a girlfriend and he said something about being friends. Well a couple days later he's like I'm going to punish you for that kitty of yours and then I told him to chill out. Now I'm worried that he's going to be all butt hurt at work and take it out on me.