When to see a doctor?

So I know they say 6 months to a year before talking to a doctor about fertility. We haven’t been trying that long, but my first didn’t take long at all. I then had an IUD for over 2 years and I’m paranoid that it is affecting my fertility. Should I still give it 6 months?