Tired of waiting

My husband and I want to have a baby. Its been a struggle because I have pcos and no periods unless I take progesterone pills to get one. I feel like my doctor doesn't take me serious when I bring up the baby subject. He just blows me off as soon as it's mentioned. I've also been having sharp pains for over a year now caused by small cyst that I have on both ovaries and he's not going to do anything to treat that either. He did prescribe me metformin but i was on that for a while last year and absolutely nothing changed. So I was thinking about going to see a fertility specialist to see what my options are because I don't know how else to go about it. My parents think its stupid to jump straight to a specialist. I feel like it's a good idea though. We've been trying for 4 year's and I've never been pregnant. What would you guy's do in my situation? Me and my husband are both 24 if that helps. We are also financially prepared for IVF if that's what it takes since our insurance covers nothing for infertility