Shouldn't we talk about it..??

Just looking for second opinion on a long winded tale...

My partner and I have decided were ready to start a family (both aged 31). I have PCOS and a recent hormone test showed my hormones are all going the wrong way for conception. Symptoms included acne, apple-shaped figure (but not overweight = BMI 21), excess hair, slightly irregular cycle. I've spent the last 12-18 months taking natural medicines, cleaning up my diet and cutting down on alcohol to try to get my body ready for bun baking.

He has variocella, is slightly overweight and is in a stressful job. About 8 months ago he slipped in work and injured his sperm machinery (swollen for a few days but wouldn't see a doc).

12 months ago when we first talked about babies, I expressed my concerns about our respective health issues and said it would (definitely) be harder for us to conceive than some of our other friends. I told him about the various articles I'd read on the matter and the advice from experts. 2 naturopaths and 2 GPs advised to start trying and if no joy in 12months, we would need to seek further help. 1 naturopath suggested my partner visit but when I asked he was not at all keen. Then when I reiterated my concerns and suggested a few lifestyle changes/vitamins and suggested we both give up alcohol for a few months, he lost it and we had a massive argument that resulted in a weeks silence by both of us. His argument is that I don't really know that we'll have trouble conceiving and that I'm simply over-thinking and being irrational. Further to this, I told him that I didn't want to tell people were trying, just in case we have trouble.... After the argument, he told a circle of his not-very-close friends about my concerns and they apparently agreed completely with him and said that I was overreacting. (Extremely upset that he had betrayed my trust like that).

12 months later, no sign of a pregnancy. We have been haphazardly trying as my attempts to be serious about TTC creates more arguments. Then, about 3 weeks ago, he sent a text saying "I want a baby". I was kinda confused (I thought that's what we were trying to do) and upset (like "I've decided we will have the baby now, why aren't you pregnant already").

I tried very very delicately to broach the subject again 2 days later but his reply was "I'm not talking about it. We don't need to talk about it". Again, I was upset but didn't let him know or try to talk again.

So now we're actively trying but can't talk about it and I'm the only one making lifestyle changes, giving up alcohol and taking vitamins etc.

It really upsets me that I can't tell him my concerns and I know this is the most ridiculous way to deal with TTC. To you, I'm sure it sounds like the most unstable relationship and that we shouldn't be trying to make a family together but we're actually together nearly 13 years (engaged for 2) and have a very happy life together. This just seems to be the one taboo challenging us.

I'd be interested to know what you think or if anyone else has experienced this reaction with a partner?

Thanks for reading