Advice PLEASE .

I was with my son's father for 7 years. we've been through ALOT. He had a bad drug addiction, he cheated on me, lied to me & all around treated me terribly... but I still stuck by him. Well. he went to jail about a year ago, so therefore I Moved on .... I'm engaged to another man now .... I've been happy with him. He does everything for me and my son .. well, My son's father has been out of prison for 3 months now ... in that time he has been clean, has been helping with our son, and actually got his own place. he has completely turned around ... totally different man . He had made it clear he wanted me back ... I'm not gonna lie ... we have hooked up a few times since he's been out ( not proud of cheating on my fiancé) but I realized I want to be with him too and be a family again ... so last night, I went to his apartment ... we hooked up yet again. feeling totally there & I made it clear to him this morning that I would leave my fiancé to be with him and he proceeded to tell me "no" because I played with his emotions for the past three months .... so now I'm completely confused and don't know what to do ... he's been fighting for me for three months now all a sudden says no ? but I know thathat I want to give us another chance.