Was I raped? Don't know what to think

Ok so this is a very long story and it happened just over a month ago. I'm a university student and over the legal drinking age. A few friends and I went to my friends boyfriends flat off campus and were drinking with his friends. I had some wine a mixer and two shots which for me is pretty standard as I have a pretty high tolerance to alcohol. Long story short I started to experience some black outs during the night, probably due to not eating properly all day so the alcohol hit me hard. 
We then apparently proceeded to go to the club and I got in, but my friends boyfriends friend text me about 20 seconds later saying come outside one of the girls can't get in so we're going. I remember this part and I remember being back at the flat but I don't remember the journey back there and it's quite a walk. 
I then remember kissing the same guy who'd text me saying we were going back but I don't remember giving him my number or getting his. The rest of the night is a blur and I remember inviting him back to mine. I don't remember the taxi ride or how I managed to get out of the car and walk down the massive hill to my place, i don't remember opening the door or anything like that. 
From this point I remember taking my shoes and my top off and oral sex happened (i gave it to him) and he fingered me. Then i remember he tried to penetrate me and i felt the pain because I am a virgin and I told him no sex, especially without a condom.
Here's where it gets odd. In my drunken state I told him to put one on because I get paranoid when bare penis's are near me. (Not that it happens often, it'd only ever happened once before) from this moment I remember us talking and laughing about how no one needed to know about our night. Then I remember him standing up by the bed after I had given him oral and just after he had put the condom on and he came right into it without penetrating me or anything. After this I was on top of him, completely fine and happy we had not had penetrative sex. Again I said I don't want a bare penis near me and told him to put on another condom if he was going to be anywhere near me. 
At this point my head started to spin and I felt dizzy so I went to lie down on my bed. He then climbed on top of me and tried to penetrate, i remember this bit clearer than ever despite being wasted. I said 'STOP *insert his name here* STOP!' And he looked at me for a second and tried to go deeper, i remember feeling his penis to make sure there was a condom on and then I felt incredibly light headed. All of this appeared to last no longer than five seconds so I just assumed he stopped. After this I called him a cab walked him downstairs and everything was fine. 
In the morning I woke up with a banging headache and felt uneasy, with constand flashbacks of him on top of me and me saying stop. I text him asking what happened and he said 'not much.' So i left it at that... Well the event continued to bother me and that particular flashback. After confinding in my best friend she told me to phone him for closure and to validate that he did in fact stop because the flashback was bothering me. On the phone his whole story changed and he said we had sex for almost 2 hours. 
 I was completely taken a back and started panicking asking how could this be and he started laughing asking if I had a boyfriend (which i don't) saying i only care because i am a girl and 'if he had woken up in the morning and seen he had been raped by a woman he'd just be glad it wasn't a man' . He said he thought i was sober snd doesnt remember me saying stop when i KNOW i said it twice and I know he was sober because he told me on the phone that 'he knew exactly what he was doing'. 
Anyway I left it as that telling him i refused to believe his story as it made no logical sense for me to remember the pain the first two times he tried it and then to not remember him in me for almost two hours. He laughed saying maybe he's small and he would not mention it to me again. 
A few weeks later he happened to be at another gathering I was at and after I spent some time with him (i was sober this time and so was he) we spoke about the night as we'd had an argument a few days after the phone conversation because my friend confronted him without my permission. 
He then changed his story AGAIN saying I have an alcohol problem (which i do not) and how I was completrly wasted that day in his eyes. I then stopped and said 'but you said you thought I was sober?' And he paused for about 10 seconds and changed the subject.
Now I dont know what to think? What if it continued on after I said stop and I don't remember? Would that be classed as rape as he was sober and admittted this snd admitted to knowing I was not. AND i said stop, twice! Sorry this is so long I just needed to be detailed. Any response would be great! I haven't got as drunk as that first time since and after spending time with him the second time I noticed he'd lied about a few things, he admitted it. This makes me think he could be lying about the two hours of sex too but he refuses to change his story after first saying 'not much' then 'only oral' then '2 hours of sex' i havent spoken to him in two weeks and have deleted his number but the whole situation has left me feeling crappy because the second time i met him he pulled out a condom expecting sex and i shouted saying i am not that kind of person. He also knows im a virgin so I don't get why he would try again. 
Anyway. I guess I just need some opinions on how to get over this. I'd forgotten the first night after I met him the second time, but some of the things he said to me in our are argument keep circulating around in my mind and now i'm annoyed for not pushing him off but i just felt too weak. The second time I met him he jokidly held down my arms to stop me moving and this triggered a flashback to the first night when he tried to penetrate and i tried to push him and he done the same thing. Held my hands down with one of his. He's 6ft 3 and i'm 5ft 10 so hes a little bit taller than me and much stronger.
If it did in fact happen, does it classify as rape? Im not going to press charges or anything as I don't want to ruin his life or mine. I would just like some validation.
Thanks for reading. 
Edit: also I am an adult and I am fully aware of the dangers of drinking and getting drunk. I don't need people trying to patronise me, this does not happen to me often. The series of events that took place were a mistake which hasn't and will not happen again. Nonetheless that doesn't permit my wishes not being respected.