Keeping baby away

Is it ever okay to keep a baby away from the father? This person has done EVERYTHING under the sun to me including putting his hands on me while I’m pregnant with his child. He’s the meanest when he’s on drugs. I’ve been staying away from him and he calls it “running away”.

I’m asking to make sure I won’t be the bad guy if I choose to do so.