Belly being too small

When I had hospital they are saying my baby is measuring fine and there isn't nothing to worry about but then I had the midwife and she is saying my belly is too small and she is starting to worry, she wants to see me before my last scan but if I'm still small she is going to make me have another scan and keep more of an eye on me. Some days I look bigger than others. Has anyone else had this ??