fathers rights PLEASE HELP

I'm in Colorado and I let the father of my son sign the birth certificate. We are not married and I agreed to stay with him at his aunts house with the baby for his vacation so he could at least spend time with the baby . I want to leave but I'm worried he won't let me take the baby and I'm worried that the cops can't do anything since he's on the birth certificate. If anyone knows anything please let me know !