Is it time to confront him?

A few years ago a bunch of friends and I were having a bonfire. I was with my boyfriend (fiancé now) and our friend(X) was with his girlfriend. She wasn't there that night though. She called me saying her sister was in the hospital then asked if she could talk to X. We both walk away from all the noise. I went with because he had my phone and he asked me to. I wasn't even thinking that he was going to do anything, his girl was on the phone panicking! While on the phone he started pulling me closer to him and touching my butt. I stepped back and told him to stop (still stayed though). He did it again so I started walking back to our group but he grabbed my arm and pulled me back to him. He started to kiss me, grope me, and trying to rub my vagina. ALL WHILE HE WAS ON THE PHONE STILL! I was trying to push him of and was telling him to stop. He's a big person(6'4, ≈260lbs). The call ended and I was able to push off of him and quickly get to my girlfriends. I didn't tell anyone at that time. I just brushed it off (idk why). About a month later, my boyfriend, X, and I all go out to eat. We go back to my house so he could drop me off. X then said he forgot his hat inside. I go with X to let him in and I waited by the door, my boyfriend is in the car still. X gets his hat and started walking back, so I open the door to let him out so he could go home. He slams the door shut and pins me against it. Once again I tell him to stop and try to push him off. I was able to move away from the door. He had a strong grip on my wrist so I couldn't get away. I kept trying to go for the door but he wouldn't let me. He started to touch my vagina. I was trying so hard to stick my hips out as far from him as I could but he wasn't giving up. His hand went down my pants and started fingering me. I was trying so hard to get him out of me. He then took my hand and forced it down it pants to touch him. At this moment I realized he wasn't going to stop until I gave what he wanted. So I gave in. I quickly stroked him for a few seconds. Then told him that's enough. He finally left. After closing the door I felt so disgusted with myself I started to cry. He texted me telling me not to tell anyone and I listened because I was so scared of him. Nobody knew for months. One day I had enough of my bf hanging out with him not knowing how disgusting he is. I finally told him, and our friends. After that we all stopped talking to him, but never told him why. I've blocked him on everything but to this day he's still finding ways to add me on social media. I want to tell him off but I just don't know what to say. Or if I should.