Not having father around son.

I've had my end with my son's father. I get it, it's my fault for having sex with him, unprotected but I never would have imagined this. My son's father has no manners, no respect, is an alcoholic that has to drink everyday, all day. He has a daughter and she is a little brat, she's disrespectful as hell and I can't stand it. I have no obligation to have my son around him just because he's his father. I want my son to have a father but he is not the father figure I want for him and all he does is call me a hoe, slut, whore, etc constantly. I just don't know what to do. I don't regret my son but I hate this entire situation. Im thinking about just moving and never worry about him again.