he still haunts me.

About 4 years ago, I cheated on my boyfriend with one of his friends. I told my boyfriend about what happened. We worked through it. We had a 1 year old at the time. Since then, we have gotten married, bought a house, and had another baby. I love him and I love our life together. 
The guy I slept with was a good friend of both my husband and I. When I was cheating, we got really close. I'd even go as far as to say we fell in love. He had (and still has) a drug problem. He would message me about every six months. Just to see how I was doing, how my husband was doing, and tell me that he missed me. I know I shouldn't have been talking to him. But for some reason, I cared about him. I wanted him to do good. 
Last month he messaged me again. (The usual 6 month mark) things were their normal "how are you doing" until he told me he wanted to marry me and he loved me and all this other bull shit.  
A few days ago my sisters boyfriend met him for the first time. (totally random place) and started talking. He found out he was dating my sister and told him about me. He said he felt bad about what happened because him and my husband were such good friends, but that he could fuck me any night he wanted. 
WHO SAYS THAT?! Who says that to someone they just met?! 
I am obviously never going to talk to him again. I never in 4 years have ever hinted about having sex again. I'm actually usually very guarded with everything I say. I just want to know that he's alive and not in jail. 
Part of me wants to message him and tell him to leave me alone and never contact me again. Part of me just wants to ignore him forever and hope he gets the hint. 
What would you do? (Please don't judge me for what I've done. I know I was wrong for cheating and wrong for talking to him at all)