Do you have to answer to CPS?

So I get a knock on my door. I'm home alone with the kids so I peek out the window and don't recognize the people so I slowly Crack the door and ask who they are. They say it's child protective services and they need to speak to me. I told them I didn't get any phone call about this and have never been involved with them before and they keep asking to come in. I said no I need proof I don't trust you sorry. They keep insisting they come in I said no I'm going to call the police if you don't leave and they left but said it wasn't going to look good for me...

Do you HAVE to let them in? They could be anybody posing as CPS to get in tm house and rob me or kidnap my kids! I'm not letting them in with no warning... But now will this hurt my 'case' if there even is one!? I'm not a junkie or a bad mom I don't know why they would be here.