Love twist

So I like this guy, and he likes me too. We started flirting when he was in a relationship, then he broke up with his girlfriend I'm not too sure of the reason why . He is such an asshole, rude etc but I see another side of him, a soft side that's hurting, that sensitivity which he has showed me quite a number of time is what draws me to him. I know being with him is toxic and the right thing for me to do for myself is to let him go but I just can't.