Am I in the wrong?

My husband and I have been trying for a baby for a few years. I talked to my doctor and we decided to move forward with a referral for fertility testing etc. They called today to schedule an appointment and the options were one male doctor (could get me in this month) and a female doctor (a month and a half wait) and I said I’d go with the male doctor. My husband and I have been fighting since because he’s not comfortable with a male obgyn and I told him that it’s not his decision and it’s my doctor and he should be supportive either way. He says I’m not taking his feelings into consideration and that I’m in the wrong. I feel very differently. Anyone go through something similar? How were you able to work through it?