Doctor changing due date?

At my 20 week anatomy scan, my son was measuring two weeks ahead. We went ahead and re-did and it’s still showing my same thing. My doctor wants to change my due date but I’m not sure. My first son was always measuring ahead and they predicted he would be 10 lbs and he came out at 6. I just don’t want the doctor to pull him out early if he doesn’t need to be. Has anyone experienced this too? Should I deny it?