Implant Birth Control

I got on the IUD Birth Control On June 13, 2011 then had taken it out on Cinco De Mayo 2016 so it's been about 6 years and I will be taking it out a day after my Birthday on June 14th 2018 to try and have a baby. My question is since I have been on birth control for so long will it take a long time to conceive?? I hear it takes a year others say right away, if anyone can tell me how long it took them that be great because me and my husband have kids from previous relationships but non together. So I'm really excited to be trying this year coming!