anyone know?

Grace
My babies 5 weeks old and his dad wants to take him to his house. We have no court order so if he takes him he doesn't have to return him. And out FIRST meetingbisnt wven until the middle of November. He wasn't being very nice today and I told him he can't come over. Can this really be used against me in court? That I won't let him see the baby. What mother would risk not seeing their baby again til court? Am I being irrational?