If you had the chance to be with the person you always wanted to be with would you take it?

No bashing please I understand I might be stupid but I need advice I have no one to turn to.

this guy (let's call him Ben) I went to school with gave me the courage to leave my abusive ex. We became best friends and always had my back. Eventually I started falling for him and apparently he started falling for me. We have been friends for awhile before we finally had sex. After that he started blowing me off eventually we stopped talking. He came and went in my life only time he came around my opinion was for sex only. I dropped anybody was talking to for him, I told him how I felt over and over and he said it felt the same but it's hard for him because he wants to be my friend at the same time he wants me more as friend. this went off and on, a little before graduation I had gotten a boyfriend treated me like a princess ben messages me saying how much he missed me and wants to be with me that he's tired of playing games and wants a life with me and promises to change. Since I already had a boyfriend I declined so we stopped talking. After graduation and I moved he messages me again saying the same thing, I declined again because I'm still with my boyfriend and I'm not gonna leave something good to be left high and dry. Well of course relationships aren't always perfect my boyfriend has been blowing me off and we've been arguing alot. He's been snapping at me and my son a lot and blames everything on me, he's 5 years old than me and yells at me to grow up when I'm trying to and we only been together 4 months. Recently ben has been messaging me saying he misses me and wants to show me he changed. I don't know what's about him that I can't let him go, but I don't expect him to wait around forever. I don't know what to do..