People saying I'm small!!

Grr it really bugs me when someone says 'you look small' it makes me worry about my baby. I love my little bump! I'm 22 weeks. Anyone else gets this problem!?