He said he'll never forgive me

I was at the bar last night with my boyfriend, and I think he had too much to drink or maybe he's just that colossal of an ass, but he decided to tell me at the bar that he can't and never will forgive me for being raped two months ago when I got roofied by a friends acquaintance at a bar while out with my friends when my boyfriend was out of the country working.

And he went on to say he's never made a mistake like that in any relationship he's ever been in, he prides himself in being faithful, and he can't believe I would do that to him etc and he was unrelenting about it.

I cried for probably an hour while we were there, and at one point I walked out into the parking lot and he chased me down saying "oh really, you're gonna be like that?".... like I was the one being a dick..

And then he wanted to go home and have sex after the bar closed at 3. I denied him because I was so pissed at him and upset.

But am I wrong for being upset with him? I think he was way out of line! I still can't believe he even said that to me.