Go or Don't Go.

So I'm going to try and be as short about this as I can. So I was with my high school sweetheart for 7 years and we had 3 kids together. Our youngest is 1. I recently left him about 6mnths ago. He's never hit me and Ino every relationship isn't perfect. We moved back from Texas the last 2 months of our relationship  and he didn't have a job which stressed him out. We argued a lot and I think we needed some times apart. So I met this guy and we've been seeing each other 4 months he has 2 daughters. He's going through a custody battle right now to get full custody. He doesn't smoke but he does drink. Ino we can't live together right now because of the battle but at one point in our relationship he didn't come around for about 5days. He would look at all my messages but never reply. Now I'm really sceptical because I seem on his phone one day we're he has a app called line and he's saying that's were all his friends talk about thier game"clash of clans". His court is suppose to be Tuesday but he said they pushed something so it's going to be even longer. As of right now I see him 1week on n 1 week off. When he has his daughters his mom will not even allow him to come outside n talk to me or even see me for 5 mins. Thier lifestyle is so different and it's so hard to get use too. Hes always accusing me of talking to someone else or comparing me to his ex wife making comments like "don't get too drunk n cheat on me" or "you better get your phone he will get made if you don't text back". The other day when I took my kids too a museum thier dad met them thier and spend some time with him well he got mad at me and told me to enjoy my date with my ex. I'm starting to miss my old life. I don't know if I should stop y it's early or just let this feelings pass for my kids father.