MFI - Varicocele

Hi ladies. DH was diagnosed with severe MFI (1.5 million total and 20% motility) in January. He had a bilateral varicocele repair 2/5/2020 and began taking clomid. 2 months post surgery he had a repeat SA and the sperm results were now 0. I am completely devastated 😭 I never imagined the count would get worse. The urologist referred us to a different urologist that specializes in fertility and said to give it more time. The thought of waiting month after month is so daunting and depressing. At this point I just want to do the IVF but it is so expensive (no insurance coverage) and now we have no sperm. I just don’t know what to do... anyone hear of a SA getting worse before it gets better after varicocele repair?