Infertility and a difficult husband

I know Husbands are not always easy going through the infertility process. I am sure many of you out there have similar stories to mine. I feel like not only am I battling infertility but sometimes my husband as well. We are about to begin our first <a href="https://glowing.com/glow-fertility-program">IUI</a> cycle and I have never felt more alone. He constantly talks about how badly he wants a baby but then is so annoyed that he has to go through any of the steps we need to take. "No one else has problems getting pregnant". I hate that comment. All I have to say is you don't know what anyone else goes through. No one sure as hell knows what we are. He keeps telling me that it is unnatural and that he has always thought it is wrong to go through extreme measures to have a baby. But then in the next breath says how much he wants one and how sad he is to see other people with their kids. By the way he was the one who wanted kids first. 
We have been ttc for 3 years now. 2 years ago we got pregnant only to end up having a miscarriage at 7 weeks. Since then no luck. We went to the specialist last year and were diagnosed as unexplained infertility. Since then I have done two rounds of clomid, acupuncture and countless old wives tails, all trying to find a way to make our dreams come through without making it too difficult for him (I know why make things easy for him when I was putting myself through all sorts of things). 
He has finally agreed to do the <a href="https://glowing.com/glow-fertility-program">IUI</a> (it has taken a year to talk him into it) but then complains about it. Like its all such an inconvenience for him. I have to say I used to be so hopeful in trying this step but now my heart isn't completely in it because of his attitude. 
I guess basically what I am asking is how other women handled difficult men when going through infertility. I need help because I have never felt more scared.