Think my due date is wrong

Hi all! I am currently 19w 5 days pregnant (according to ultrasound)

I had both my 7 weeks and 12 week scan and should have been around 20 weeks 4 days but then a few weeks after 12 weeks I had an amnio when the obstetrician changed my due date which made me 6 days earlier.

Here’s the thing, looking back at conception dates my husband and I had sex on the 5th/6th of the month, after this we were apart until the 15th.

The conception date according to my ultrasound due date is 12th (not possible as we weren’t together) I took the pregnancy test on 20th and got a BFP. So... my question, my husband and I had sex on the 15th when I got home but he pulled out this time (we weren’t meant to be trying as I have a chromosome defect that can affect the baby) but on the 5th/6th he did come inside.

So, is it possible that a pregnancy test can possibly test positive after 5 days only ? (Even though he withdrew)

Or is it more likely from the 5th/6th?

I ask because I worry maybe I should tell my doctor I don’t think my due date is correct, I worry the baby is small 🙁