Dating Advice

So I dated this guy about three years ago for a while. I broke up with him and immediately regretted it. We kept in touch as friends..with benefits. I know I still really like him and care about him and he visits me by surprise and is often In touch with me, but recently he told me he doesn't want a relationship and that things between us should end. I was very upset to hear this. Then he keeps messaging me and I reply. Now it's all on his terms..he messages when he feels like it and I reply. When I message him or ask him something he doesn't always reply until later. Keeps me waiting. But when I see him he kisses me and is very affectionate. Sigh..do I come to terms with this and let the one I really like go or let him come around when he is ready? What should I do?