know wasnt rape but

i have never told anybody what happend but since a few month i have to think about it more often. maybe because je is in my life more often again.. but lets start from the beginning. when i was 14 i had a best friend who was 16. we did a lot together but i never considert sleeping with him. maybe i was naive or something. one night i had a sleepover at his house. we slept in the same bed and when we stopped talking and where about to go to sleep i felt his hand going up my leg. i pushed him away but he did it again. after i pushed him away a second time he tried to manipulate me with words and it works, I did ot push him away again and he fingered me. it hurt but I didn't say anything anymore. that happend on another day as well, kind of similar. finally we departed because of a fight. I could always deal with it as my first sexual experience even if I didn't want it to be. but now I'm 20 and in a relationship with his best friend and I really love my boyfriend. but the thought from ages ago come up more frequently now that my former best friend is back in my life again. but I can't tell my boyfrie d because I don't want him to choose between me and him or lose his best friend. but I can't get the thoughts out of my head. I know it wasn't rape but I still feel violated

Update: does nobody has any thoughts about it?