When is it time?

Shantel • ShantyAClaire ❤️👑 I like to write what I feel.
I've been on and off seeing this guy for a little over 3 years and I feel that it was going nowhere. When we met, we went on a date and it didnt work out. We stayed in touch and a summer later, we started sleeping together. This went on for a couple months and then we lost touch. 
Fast forward to October of last  of 2016, I just got it of a relationship and so did he. We basically continued where we left off. I don't remember sex being so amazing with anybody. We just connected on every level, mentally, physically; it felt so right. We weren't in an actual relationship, but we did everything together. We were more like best friends, we told each other thing that we never told anybody else. I've never had a guy tell me and what made him insecure and I've never told anybody about what made me insecure. He always told me how beautiful i was and he made me feel beautiful. We were together on birthdays and we spend Valentine's Day together. So why didn't it work? Why didn't he claim me as his ? How do you tell a guy that calls you his best friend that you are in love with him? 
On New Years he got drunk and told me that he loved me, and didn't want to lose me...so why didn't he try to keep me? Why did he let me go? I told him that we needed space, that I need space. Why didn't he try to stop me? He said he will respect my wishes if that's what I want and asked "why"? He was the best thing to ever happen to me, so I'm asking myself the same question. Why couldn't I tell him how I felt? Why couldn't I just say "because I'm in love with you and I don't know if you feel that same." Why couldn't I just say it? Because I didn't want to lose my best friend. I asked him if sex was the foundation of our friendship? I asked him if we could still be best friends without sex? I seemed ok with the idea but then he told me that he was going to go and that he would talk to me later. Would he even talk to me again? Is there something that he wanted to say but didn't? I just don't know what to say when it comes to him. Should I walk away or should I tell him?