Mirena removed after 5 years and ttc

I had my merena taken out in November. I was having protected sex but my period was not regular since my body was getting used to not having one for 5 years. In April we started to ttc, my periods had came either 29-31 days. In July i had a 18 day cycle and again this time my period came a 20 day cycle. We have been having sex every other day and then I will start spotting when I thought I was ovulating according to the opk. Has anyone else had problems ttc after the mirena? And how long should I wait till I go see a doctor? This is very heartbreaking