Uterus measuring small

I had my last ultrasound at 19+4 and baby was measuring 19+3. I am now 24 weeks, Yesterday I passed out at work and since I fell I went to the ER just to make sure everything was okay, the doctor said my uterus was only measuring 20 weeks. Should I be concerned that my uterus is measuring so small??