I want to ask him out but things in his life keep coming up

I never in my life thought I would be open to dating. I've always hated and been afraid of men because of my past. I'm a victim of SA. I attend a support group for it. Last year for the first time we had a man come to our group and he freaked me out. It was always only women and he never really spoke at first. I just knew he was there because he's on probation and this is part of his probation to attend this group. He he had a baby the counselor had to sign. He made it clear that he thought this was stupid and he didn't want to be here. I had one day gotten upset with his lack of participation and said some hurtful things. I thought maybe the crime he committed was raped and that's why he's here. Looking back yeah that didn't make sense why he would be sent to an SA support group. Like I said. I said some pretty bad things and I found out why he was forced to come here is because he got arrested because he was in a fraternity one one of his fraternity brothers assaulted him so he was in shock and drove his car through the house. I got in trouble with the counselor for how I treated him. I apologized at the next meeting. And... We kid of are best friends now. We started being there for him. We both have PTSD. Well I have CPTSD and he's been great with me. He's bipolar ll so when he's having his lows I've just laid in bed with him to make him feel better. I have started catching feelings. We hang out almost every day now. Even after he fulfilled his SA support group hours(not his probation. He's gonna be on probation for 3 years) he has still continued going to support group. I've been wanting to just ask him out myself but there seems to be something going on every time I want to and I don't want it to come off that I'm taking advantage of that. He was recently spiraling because he feels he ruined his future about college and his fraternity. Then he was going through something with his family. Then he had some issues with his former first brothers and whose "side" they're on. He's been going through it and just wants to come over and listen to drama podcasts together. My cousin thinks maybe he doesn't need a girlfriend right now. Maybe he needs a friend. Should I just not ask him out at all for right now?