Should I find a new OB?

I have been TTC for about 8 months.  Discovered I was not ovulating after about 3 months, my MD prescribed metformin. After two months that wasn't working. Did an ultrasound to rule out cysts, etc. Everything was fine, so we tried letrozole for two months with an ultrasound each time. When aunt flow started the second time I called and asked for a refill and the office said no, that I need to go to a reproductive endocrinologist! They didn't give me another round so we could try again in the meantime, so I'm out for the whole month of July. They wouldn't request a semen analysis for my husband either. Am I wrong to think that two rounds of meds is ridiculous? I feel like they're throwing me under the bus. "This one can't get pregnant, get her out of here." Plus I told them there is no way I can afford to go to the other place. I'm barely able to pay for the office visits I was getting. I'm so frustrated. The worst part is that the letrozole was working! I just need someone to prescribe it for me and I know it could make all the difference! Any suggestions are greatly appreciated. Should I find a new office or just suck it up and go to the RE.?