Conception date not matching up to 1st scan?

I know the date of conception, zero doubt. Or at least the date the "lucky deposit" was made. My period was 4 days late & still only got a faint positive, the next day a negative from a digital, two days later pos blood test.  Based on LMP, EDD was 1/25. At 1st doc appt, during scan doc said I wasn't anywhere near that far along. He gave me a range all the way up to 2/2 for EDD. He asked if there were any other possible dates of conception, I showed him my tracking app, (2 times that month & one was way later in month...yeah I know, we were boring that month) & he agreed, May 4 had to be the date of lucky deposit. So we settled on EDD of 1/27. Now here I sit, almost a week overdue, & he's concerned about size of baby & wanting to do a c section. He doesn't like allowing patients to go over 41w, but what if I'm actually only 40w & 2/2 is a better fit? I don't want to rush things (as freaking miserable as I am) & be cut open for nothing. Has anyone else ever had this happen? Thinking I just ovulated late? My period the month before was also late. Just confused & don't want unnecessary intervention, but also don't want to attempt to birth a 10 pounder with my small frame.