Conceiving after mirena?

So I had the mirena in for 5 years. I got it taken out on November 19th, 2014. My husband and I have been ttc since then and have had no luck. I want to know if anyone else has had this problem? I'm trying to be patient but we want this so bad!