Testosterone replacement killing sperm?

DH has been on Androgel for about 18mos now due to low T. I've researched and many couples agree Androgel or testosterone replacement therapy can hinder chances of pregnancy. I asked DH to inquire with his PCP; PCP told him "nothing to worry about". Has anyone been in this boat and opinions?? (DH will be getting an SA done very soon; I have 1 child he has 0)