Confronting my rapist?!

A couple of weeks ago I realised my sexual relationship with my ex-boyfriend was partly him raping me. What I mean is that there were times when we would have sex and I didn’t want too/didn’t consent, because the one time I said no he got scary and so annoyed at me I never said no again. At the time I thought it was okay because we were in a relationship, only now am I realising it’s not. I’m constantly afraid of running into him so I thought it must be best to confront him face to face and tell him? I don’t hate him but I feel like he needs to know what he did- but I don’t know how to tell him or what to say. Any advice would be much appreciated!