Tired of trying!! Anyone else?

My husband and I have been TTC for over 2 years with no luck! I have had two surgeries for uterine fibroids in '12 and '14 and started TTC 3 months after the second one. In December ('16) I had an HSG and was given the all clear (even though I do have 4 new fibroids). The doc said there is no reason why I shouldn't get pregnant. My dear husband however has not gone for his sperm analysis and gets upset when I bring it up. He finally admitted to being nervous that he's the reason we can't get pregnant and then goes on about how we can't afford a child right now, how we live in a 1br apt etc... But then will go on to say he wants me to get pregnant. 🤔😟 I've gone through so much already and every month is such a disappointment!!! I'm so tired of trying! If it weren't for the continuously growing fibroids in my body, I'd totally be fine waiting a year or two. We are only 34! Just need to be grateful to have each other and out little kitten!