Semen Analysis :(

We just got the news that my husband has low sperm motility and there is a deficiency in sperm morphology, none of them are shaped correctly and are very slow. The doctor recommended clomid for my husband to boost his testosterone, levels which were also low. I am devastated so is he. The doctor said he can try the clomid for 3 months with 3% success rate. After that he said we will have to consider IVF. What do we do now? Do we try the clomid? Should we get a second opinion? Do we just do IVF right away? I feel like crying :(