What can cause a low sperm count?

My boyfriend and I have been ttc for about 12 months now but he doesn't smoke or drink he did smoke weed before but it was a longgg time ago before we even met also had depression back then and was put on anti depressants and was taking them for 6 years but now he's fine. I asked my ob gyn if the antidepressants can cause problems with sperm count she said no. He's not stressed, he doesn't drink caffeine at all, and he's at a healthy weight he doesn't have any issues ejaculating. We're trying to just give it 2 more months if nothing happens. I honestly think he's fine and I'm the problem hoping its just stress. But can some men have a low sperm count and have no idea why?