2 years after depo...

So my last Depo shot was September 3, 2014. It's been 2 years since my husband and I have been trying to conceive and still have had no luck. 😭😭 My periods became regular about 4 or 5 months after the shot. I've had blood work done, ultrasounds, numerous doctors appointments and everything always comes back fine. My doctor says the reason I'm having trouble must be the shot and to be patient and give it time. It's been 2 years!! Is that not enough time!? Ugh it's so frustrating and discouraging. My husband hasn't been tested in any though. So now I'm staring to wonder if maybe it could be something to do with him? Or if it really is from the depo shot? Is there any supplements that he can take to maybe up our chances? Or should I just give it a few more months and see what happens?