TTC after BC?

I'm on the pill but after about like 8-12 months me and hubby will try to conceive again. I've heard that it's difficult to conceive after stopping your birth control. Have any of you guys conceived as soon as you quit birth control or did it take a while?