About 6+ months ago I was nearly assaulted in my car after giving a coworker I was flirtatious towards a ride home. 

I talked to his sister about it because she was my friend and I told a girl who was also talking to him that I didn't want to talk about him anymore because something happened between him and I was uncomfortable. 

What happened exactly? 

Well. It was late and we got off of work. He can't get his license in this state and I don't know why but that's why he doesn't drive according to him. So I drove him home like I usually do when we worked together because he lives close by. But that night we sat and talked for a bit. He was mostly talk me about his family and his dogs. He kissed me at some point and I wasn't stressed about it I kissed him back it wasn't a big deal we were flirty. But he really aggressively tried to get my pants undone and I said no a few times and pushed him away and after awhile he said "what's wrong? You want it." And I told him then to get the fuck out of my car because that's how rape happens and I'm not playing that game. 

I asked his sister when I spoke with her if I did something to cause the unwanted advances or if maybe he was just like that or something and she said he had gotten himself in trouble another time with some unwanted advances to another girl. And I said to her that I felt like it was boarding on assault. 

I've never had any intention of going to the police or anything because I handled the situation. I've since left the job and moved on as well as deleted and blocked his number because I don't want to talk to him ever again. 

According to a friend I have there still, now someone told him I said he tried to rape me and I'm not sure how to handle it. It was obvious to me that night he does not think he did anything wrong. In addition, someone lied about me before at that job and he said I was obviously lying about it because his source was reliable. 

My question to everyone is should I ignore the situation, do nothing, and let it pass or should I try to talk to him about it and convince him that he was wrong but I would never call something that isn't rape a rape? 

I'm not here to ruin his reputation, make him lose his job, or anything. What happened is mine to deal with. I just don't know how to deal with it.