Trying to get pregnant after having the mirena

I have been off my mirena since February 2016. My boyfriend and I haven’t used protection at all after having it removed, but I still haven’t gotten pregnant. My doctor told me I could get pregnant as soon as one cycle. I’m not sure what is going on. I’m not sure if the issue is on me or him. I have had a kid before and had no complications with my mirena. It’s hard to track my ovulation because my cycle isn’t regular. It was for a couple months, then changed.