Is my daughter truly missing out if I deny her dad access to her?

Backstory — her dad was never in her life. He dipped out when I found out that she probably wasn’t going to make it at 23w. Wasn’t around when she was born, around 3/4 months I officially moved back into the city and he only came around to try to rekindle our relationship. Which ended up causing drama with his then wife. An altercation happened and he called CPS on me lying and filed for ex parte custody just to let everything fall through. He however signed my protection order against him. Missed the following court dates for custody but then tried to kill me because they dismissed the case. Had females harassing me at work & popping up at my house in the middle of the night looking for him and claimed to have tried to kill himself because I didn’t want to work things out. Had him move out of state thinking that would make him want to be a better parent & it didn’t. He left caught a felonious possession of a fire arm & told me, me and baby girl wasn’t safe in the state he was in when me and the baby came to visit. Since then he has hit me multiple times while I was holding my daughter and choked me out in front of his other child. I ignore him because he doesn’t call to speak to her, just texts me saying that he’s not happy with his new bm. His parents believe that my daughter will be the one missing out if I keep him away from her. She’s traumatized and I’m suffering because I don’t know how to help her outside of cuddling her. I don’t want him around at all, but they are fighting me on this.