What should I do ?!

So I’ve been with this guy for a little over a year I was not taking him seriously it was more like a fling because of certain situations that didn’t make sense to me... so I’ll start from the beginning when I first met him he asked for my number while I was at work he didn’t look like a bad dude and he was very handsome so I gave it to him... we started texting then a month later we started messing around... this is where things started not to make any sense we would always go to my house I could never come to his but I didn’t really think to much of it until a couple more months down the line I get a text from his kids mother telling me that she’s known about me for a while a they live together because of bills and their kids... he told me about his kids and that they lived with him but I kinda knew about his situation. I was close to cutting him off because I thought he was messing with her because I’ve never met a man living with the mother of his kids and it’s nothing going on.... well she confirmed he was telling the truth... so I began to take him seriously... a day or two went by she’s texting me asking can he come stay the night with me and I’m confused at this point because why are you still texting me you guys aren’t together... she basically didn’t like the fact he moved on & did everything in her power to get him to leave and to get me to leave even threatened to take his kids from him... so she moves out and I started to come over.. I asked him is he sure its okay for me to come over I don’t want to be in the middle of anything he say yes she knows you’re coming over so I left the situation alone. But he’s peeking out of windows and doesn’t want me around his kids because she doesn’t want me around them... now she’s going around telling people im a home wrecker... & today might be the last straw for me. I’m over his house and the door bell rings and it’s his bm so he goes to the door and she wants to come in but he won’t let her now I’m confused 😒... she gets mad and leaves.. I have nothing against her I honestly felt bad she was made to stay outside... I was even told she would try to fight me... because of him being with me and not her. How should I go about this situation ??