6 week wait

So I had a miscarriage at the beginning of the month. I saw a new doctor after sincemune was booked.  He told me that once the bleeding stopped we could start again. But when I went and saw my usual doctor she said I shouldn't have sex for at least 6 weeks so I'd have to wait until December 16 to try again. My first doctor said women are often more fertile after miscarriages and pregnancy but my other doctor said I should wait. But I've already ovulated and had sex on that day. Am I okay to still have sex or do I really have to wait 3 more weeks..??  Thanks