Low sperm count..urologist or infertility specialist?

My husband has a low sperm count, they didn’t even give us a number. We are currently working with a urologist who just ordered a blood sample to test his testosterone. Should we go straight to a infertility specialist??