Sperm count dropping after VR??

Desire

Hey everyone, feeling discouraged and needing answers. My hubby had a VR in March. First sperm analysis done in August with numbers at 4.5 million. Kept taking the fertility vitamins and just took another SA in December. Numbers came back lower... 2 million. All the doctor said was to keep taking the vitamins and also incorporate 2 Aleve everyday? And new SA in 2 months. I feel so down and hopeless. Has anyone else experienced this? Any advice?