Advice please. Its really affecting me

These last few months i have made contact with an ex. He isnt just any ex, he is the love of my life. He is also my first love. Now i am aware that people say you never stop loving your first love but you do move on. Well i was "moving on" for the past year. But something just was missing in my life. So i called him. I didnt expect and answer or any future contact but now here we are two months later and i have completly fallen in love with him. The problem is in the beginning of us talking again he told me he doesnt see a future with me, he doesnt want a relationship and i was perfectly ok with it. But then certain things were changing. He was asking me to got with him to family events. He introduced me to his friends. We talked every day. Hung out every weekend. He even asked me to move in with him. But then when it came to talkimg serously he still kept saying no i dont want a future with you bc of our past. Now its gotten to the point where i cant take it. I suffocate when i teven think about losing him in my life but i cant handle him seeing other women. Truely i dont know what to do. I love this man with every inch of my heart.  And he does love me. But i cant change my past. And thats not fair.