E-cigs and sperm health

My DH has two older children from a previous marriage but we have been TTC for about 1 year without success. I have had thorough evaluation due to an autoimmune disorder but they think I should be fine to conceive. We have tested my husband's sperm 3 times now and each time worse than before. He had been on testosterone replacement which he stopped months ago because we know it can worsen parameters. The only other thing is he has been smoking e-cigarettes for about 4 years. Our fertility doc says that nicotine can have some negative effects but she isn't aware of anything specific with e-cigarettes. Has anybody had any experience with this? At this point we would be lucky to have <a href="https://glowing.com/glow-fertility-program">IVF</a> with ICSI be successful if we can get his numbers up a bit...