pregnancy problems

So I had my daughter October 13th 2013 after 2 years of trying. I got on the mirena in December that same year. I had my Mirena removed 2 1/2 years later in June 2016. My daughter will be 5 in October and it's been nearly 2 years. I don't know if the problem is my thyroid because when I was trying the first time I had just gotten on my hypothyroidism medicine properly in September and found out I was pregnant late January. I suck at taking medicine so I'm not sure if my inconsistency with that is what causes me to not get pregnant or my husband is now worried that he is the cause of our fertility issues and wants to get tested. Ether way I turned 30 March 26 and I don't want to be 40 before I have another child.