I feel in love with my 3 night stand.

A guy I've know since I was 15 years old and I here recently slept together  twice this month. We did 2 years ago as well. We've always had flirty attraction going on and when he proposed to have sex here a month ago, we ended up meeting up and I had the best sex I could ever imagine. Every emotion building up to that point was released and it was like a real life version of 50 shades of grey. 
I HATE to admit this because I know there are people on the prowl waiting to attack back with negative comments. BUT, we are both in relationships. I feel like we are both yearning for more outside of our relationships because our current relationships lack something we see  in eachother.  
I started falling in love with him. I think it's because before the topic of meeting up for sex came up, we were just causally texting venting life's problems to one another and he told me he loved me once. Said he was genuinely in love with me and wishes things were "different" . we had sex and it was great. but then he was becoming more and more distant and started to maybe like me less each time we had sex. It's gotten outta control. It's not like me to be Disloyal. I guess the heart wants what it wants..... How do I get over this?