please don't bash me

Sorry for the long post anyway
Me and baby's dad split while I was pregnant because of him being violent, after that I knew I didn't want my baby growing up without knowing their dad so I gave him all the dates and times of scans and appointments etc, never turned up to 1 of them even though he doesn't work and just lays about all day so it wasn't due to the fact he was working and couldn't make it, all through out my pregnancy he made up lies and rumours about me, kept coming up to me and threatening me, he even pushed me in the stomach one day, anyway babies born, dad doesn't come to the hospital even though I let him know he was born, I then found out he was doing weed and cocaine so I refused to let him see the baby, it's now been 7 months and aside from trying to demand a dna test (definitely his) he hasn't tried to see his son even though he knows where I live etc, hasn't offered me 1 penny (even though I wouldn't take his money' I keep getting messages and calls of different members of his family and friends saying things like I'm not fit to be a mum because I'm stopping him from seeing his kid and that his mum is going to take me to court and all this, do you think it's unfair that I've stopped my son from seeing his dad who is so violent he used to punch holes in the wall by my head, he gets arrested countless times for fights and being drunk, is on cocaine and weed and he also tried to keep me hostage in the house once, he loses his temper at the slightest little thing, I told him I wasn't in the mood for sex once and he went and punched a hole in his door, am I being unfair?