Co-Parenting

I have a one month old. Me and the father broke up after I saw text messages in his phone telling Another women he was going to get him pregnant. I allowed him to see the baby when he wants but I did not allow him to take the baby. Now he’s refusing to see the baby unless he’s able to take the baby where he wants. Is the baby to small to be with out his mom?