I Slept With My Best Friend's Ex Boyfriend

Allison
so, i've violated girl code a lot with this but i've been sleeping with my roommate/best friend's ex boyfriend since the end of december. they were together for 5 years (and broke up in august or september 2016, it's been about 6months), had a child together as well and he comes and stays when he doesn't have a sitter or she doesn't have a sitter for her 3rd shift job. what even got anything started was christmas eve. she had to work and he came to watch both kids (she has 2 kids, each different dads). things were innocent up until he put the kids to bed because he grabbed the icing we had used for the gingerbread house and started an icing fight where he then ended up kissing me. from there we started texting and the next time he came over we ended up having sex. and it's been a thing since that when she isn't here he kisses me and we end up sleeping together. i really like him, and he even said he likes me and wishes things could be more than what it is right now. but she's threatened to fight me if she ever found out i was sleeping with him. 
am i in the wrong? am i justified? anyone else ever been in this kind of situation? 
edit: okay, yes, i know i'm wrong but at the same time she did plenty of messed up things while in a relationship with him. and it really wasn't intentional for this to happen. i feel bad about it but also i see it like he's his own person as am i and sh*t happens.