HELP please it’s a long one

So basically me and my ex (daddy of two kids 3&8 months) have officially broken up for good. He’s refusing to see them unless I’m not around (yes that’s fine) but it isn’t fine for him.

My ex is an alcoholic, he walked out on me the whole 2 times I’ve been pregnant and comes back once baby is born. he’s mentally and emotionally abusive. Didn’t realise he was till I tell people and speak to my councillor about him n the things he has said to me. He’s currently on probation for drink driving..

My ex lives with his mum and 2 brothers. One of his brothers at age of 14 is on the sex offences list because he abused a girl who was 6 years old - a reason my children aren’t allowed in his home alone or at all.

Now my problem is my ex saying he’s taking them to his home and if I don’t he will get me done for domestic abuse (only ever slapped him when he cheated on me with his best friend) so I don’t know where I stand he said he isn’t going to stop till he takes me children away and he’s even making up lies that I beat my children even though I’ve never harmed them in any way.

Can anyone help please? I’m having sleepless nights over the situation and how he can lie about it all. I’m heartbroken