Should my husband be mad at me?

About 5 years ago I had a threesome with my best friend and her boyfriend. They are now engaged, have a 4 year old son, and I'm supposed to be in the wedding in May. Nothing has ever happened since. My husband and I have been together for 3 years so this did not happen while we were together. He asked my last night if I had ever slept with the guy. I told him yes and he freaked out and said he wanted a divorce. It broke my heart especially since last night was my birthday and we had had such a wonderful evening. Today he said he doesn't think he will ever get over it but we could try and work it out. He also said I am not allowed to be in the wedding and if I continue to be friends with the girl then he was filing for divorce. I am just shattered I don't want to lose my husband or best friend. My husband and I also have a 4 month old son.

*Edit* yes he asked me outright about it. I have no idea why. He made some weird comment a couple of weeks ago about the guy and I didn't understand so now he's accusing me of lying the first time he asked. I told him I didn't even understand what he was talking about a couple weeks ago but if I was trying to hide it I would have told him straight up last night when he asked. I have never cheated on him and I love him so much. I just feel horrible and hurt. I also didn't ever mean to do anything to hurt him. :(

*2nd EDIT*

This girl and I have been best friends since Jr. high so this is so hard. I'm supposed to be her maid of honor. Her and I have grown up together, raised our kids together, been there through every rough time. The thing that I don't understand is he has an ex girlfriend that he talks to all the time. She lives in a different state and is engaged, but isn't that like the same thing? I actually feel like that's worse because they actually had a relationship. This was a one night drunken thing that I wish more than anything I could take back. My husband and I are doing OK. He was being pretty nice when he got off work last night and he acted OK this morning. I just don't know what to do.