How accurate are 20 weeks ultrasound?

I'm at the doctor office and my doctor is saying that my baby is not at the growth rate that he should be at. Im 21weeks and 5 days but my baby is measuring at 20 weeks and 3 days. Is my baby not growing or could I have conceived later than I thought? I'm waiting on the doctor to come back but I'm really scared right now, has this happened to anyone else? 
At my first ultrasound they gave me the due date and approximately how far along I am. Could that date have been wrong all along or is it something wrong with my baby?
Please help!