Should I be worried

Kayla • Expecting baby girl due November- IVF #3 FET
My husband and I have tried to bank sperm three different times- and now we are going again the end of the month. 
The nurse coordinator called me and said after the semen analysis, they will "consult" with us. 
We are dealing with low count (since April, 39 mil, to 5 mil in June, and 2.5 mil in July) and 5-0% motility(5% in April, 0% in June and July).
Is there any possibility of them telling us we can't do <a href="">IVF</a> with my husbands sperm? Should I be preparing myself for this news??