Disappointment at Doctors visit

Today I had an appointment where my baby was measuring in the 90th percentile I’ve previously had a perfectly healthy (no GD) pregnancy with a large baby. I’m plus sized around 245lbs pre pregnancy and I haven’t gained any weight only lost about 15lbs, I was the same way with my 1st only gaining about 4lbs up until I gave birth. I have this doctor that was saying my baby needs to be more closely monitored because I’m a plus sized pregnancy and that’s why my babies are so big. I just find this offensive and not medically accurate but I’m not a doctor so maybe it is. Have any of you heard of this?

Edit: Also I want to add that I know a higher weight gain in pregnancy and result in a higher birth weight of baby but I’m loosing and gaining only a few lbs by the end. That’s why I’m confused on why he said and plus sized pregnancy results in bigger baby.