Low sperm count

Feeling really discouraged after my 2nd IUI. My husbands sperm count was even lower than the low count from the first (0.49 million) he’s been taking all the recommended vitamins for almost 4 months now so I am so confused why it’s only getting lower. Does anyone know why this could be? His motility is at 90% but the count just seems to get worse