Sex-friend with mental illness

I am a single mom and I just turned 40. I left the abusive father of my 2 kids last year but the relationship ended really 4 years ago. I have now this idea that getting into a serious relationship with anyone would be a danger to my kids. Therefore I am only considering Sex-friend status when meeting a new person and I make no secret about it. So far I've had 1. That is, before the events which started 3 weeks ago. A younger man (34) sent me a friend request via social media (not a dating app) and because I have known his cousin for years and we seemingly had shared interests, I accepted. We talked a little using messenger. Turns out he lives in my area. He was eager to meet. And we did, in broad daylight in the middle of the street. We talked some more. He said something about wanting to know me more, about being the kind of guy who prefers steady relationships. I told him I had no space or desire for such a thing as a steady relationship. Getting to know each other, maybe becoming friends/

With or without benefits was all I had to offer. He wanted me. I was smack in the middle of my fertility window, so hormones were driving me mad. I thought being clear about things was enough and we did have sex. That was just before I realized he was on meds. He wouldn't tell me what they were for. First shot I got at reading the names, I looked them up. Xanax and Risperdal. Contacted his cousin who confirmed he was mentally unstable and had a history of clinging to people, to say the least. I urged his cousin to talk him down, he was beginning to obsess about me, about holding me and about intimacy with me. I got scared by his intrusive behavior and put some distance between us but that only sent him through the roof. He proceeded to stalk me, harass me with Unknown caller ID repeat calls. Harassed people in the area for a clue of where to find me. And more. I'm saying all this because I have made the mistake of sleeping this guy several times. Even once would have been one too many, given what I know now about his psychiatric condition. However I am confused because staying away from him seems to be the only way I can hope to protect my kids from his erratic behavior. But staying away from him also is making him more likely to obsess more (his actions have proven that, so far) about me. And I am ashamed to admit, I miss the intensity of the physical contact with him. We did have a special chemistry that I have not found elsewhere so far.

I feel that a person should not be denied closeness and sex just because they are mentally ill but how is it possible to be close to someone like that (as a close friend with or without benefits but he'd rather insist on sex from what he said recently ) ? If I had no kids I probably would take a chance on him but I have my kids to think about... If anyone has experienced anything similar or knows of friendships or relationships which have been successful in spite of mental illness (more specifically personality disorders/borderline )...