Trying to get pregnant after depo provera

I recently had my first depo provera shot, after coming off the pill because I used to forget to take it..oops! Since having the shot, my husband and I have decided we want to try for a baby. However I've heard some horror stories about your fertility takes ages to come back. Have any of you ladies had depo provera and then tried for a baby?? If so please can you let me know your experiences? Thanks x