An abusive ex and a child in common

So I was with my daughters dad for six years several times when we would argue he would take it to the next level by putting his hands on me. I tried to stay and work things out for the sake of my little family but I finally broke up with him in December of 2013. He also has a lot of issues keeping employment and it was really hard to deal with. I am with another man who I have been with for a year and a half and I am in love but my daughters dad is so manipulative I end up feeling sorry for him. How do I end things for good but still keep a good co parenting relationship for my daughter? He is one of those types that just doesn't want to let go.. I don't know what to do.