Ready to give up...

My husband and I started TTC just before we got married a year and a half ago. I had a miscarriage almost 3 weeks after our honeymoon. Ever since, I have been an emotional wreck and I'm still trying to be positive. About 2 months ago we found out my husband had a low sperm count. He was referred to a urologist and has yet to make an appt. Anytime I ask about him making an appt he gets mad. I don't know what to do. I've asked him if he really wants a child and he just tells me to "shhh". I'm not pushy. I try to be intimate on a regular basis to just get pushed away. We are down to once a month, and thankfully that's during the week I'm ovulating. I just don't know what to do. I feel like his actions are telling me he doesn't really want this. and if that's the case I need to make a decision. I just want to make sure I'm not over thinking this.