Pregnancy after IUD and pills??

I had a son in June of 2011. Right after I had him I got the IUD Mirena placed. I had it in for 5 1/2 years and got off because I didn't feel healthy with it in. I got on the pill right after cause I wasn't ready for another child just yet. I stopped the pills on the 8th of this month (10/8/17). I want to know if anyone has had trouble conceiving after Mirena. I've heard horror stories and I'm a little scared. I'm excited about having another child even tho I have 2 already (10yr and 5yr). I am 30yrs old. 🤗