Need some serious advice!

Okay me and this guy have been going together for almost 5 years now. We have a beautiful baby girl who will be 2 in November. Well we use to get along soo great. Went to see movies, out to eat, play games, everything. Well he got into a deep depression and started doing drugs and that made us more distant, we've had our break- ups off and on. But I've always loved him..anyways the last time we ended up leaving each other I was moving into a new place he moved his stuff out and I was working on being there for myself and daughter. As we weren't seeing each other or not talking I started talking to an old friend and we start getting back in touch, well long story short we had sex and the guy I've been dating finds out and says I cheated on him. but in reality we was not together his stuff was moved out and we have not talked. Well we have gotten back together since then but he keeps gettin mad like he doesn't trust me so I'm asking what should I do? Work things out or leave....I'm so confused. because the sex was not intentional I swear it just happened, I know I could of said no but my emotions about me & him not being together made me feel like he didn't care so I said forget it and did it. I didn't expect him to take me back either but HE was the one begging me back. So again what should I do.