Really don’t want to be with baby father anymore

I feel like I’m forcing myself to be with him, he doesn’t like my family and my family doesn’t like him. He has done many things in the past that has broken my trust. I feel like he for once actually wants me but at this time I don’t want him. We are getting along and I’m playing it cool but deep down I want to be single. He’s cheated and flirted multiple times (almost caught an std while pregnant) in our relationship and I continued to stay faithful but now I want to be myself again I have forgot my life outside of him and our relationship I accepted things I know I should’ve. I was never really “taught” what a healthy relationship is because my father did the same actions as him. I don’t want to hurt him as it may sound crazy but he’s finally happy with me and I’m not.... how can I break it off? When should I break it off? We been together almost 3 years and we have a 1 year old.