When to see a doctor?

Delaney
I had my Mirena removed April 4th this year. We are almost done with month 5 of TTC. I know that isn't as long as many people and I am trying to remain positive but it seems like it's never going to happen. When did you guys go see your doctor about fertility? Anyone else get annoyed when your friends and family tell you to "just relax and it'll happen"?