Is it sexual assault?

A past fwb came to visit me and I made it clear several times that I did NOT want to have sex. We ended up kissing and he kept wanting more so I gave him a hand job basically to stop. I told him the reason I didn't want to sleep with him was because I had feelings for him and while he had 4 girls on a list of who to sleep with I didn't want to be on his list. He had just come back to the country after his vacation and we hadn't seen each other in months. In the morning we ended up having sex after I finally gave in. I looked up the meaning of consent and it doesn't seem to be this, but I don't want to believe that a friend of mine became my assaulter. Since then I can't talk to him or be in touch with him at all.