Nervous About Baby Measuring Too Big + Glucose Test

Mae
Hello Everybody,
I'm booked in for a three hour glucose test this Sunday and my 24 week scan. At my last appointment my doctor was concerned that the baby is measuring two weeks too big and thinks I might have gestational diabetes. 
I have never had problems with my sugar and my first baby was born normal weight. 7 lbs 7 oz (3.5 kgs). 4 of my mom's 5 babies were all big babies and she never had gestational diabetes.
Am I being paranoid that I might be diabetic? Is it normal to have a larger baby? Is there anything I can do to not have a big baby, even with the constant hunger atm?