How do you walk away from a toxic relationship :(

The title says it all :( 
Been with my daughters dad for 17 months now and it's just been the worst...  He is great during the week but as soon as the weekend hits he could careless about me.. 
Our relationship has always been a rocky one even when we lived together,  he left me when I was 3 months pregnant due to this weekend stuff cause he would do it when he lived with me... I barely gave him another chance after him begging a million times last month and nothing has changed when I call him out on it... The blame somehow always gets put back on me :( 
When I do nothing but stay home and take care of the baby. Mind you she's two months and he's only seen her 2 times he has never even carried her