Tricare & Fertility Testing

My husband and I have almost reached the year mark for TTC, we're young and the fact that it has not happened yet has become concerning. I am on prime and have to go to naval, I've only been once to my doctor for allergies about a year ago. My PCP was not exactly "nice" the last time I saw him, and I'm nervous about talking to him about my concerns. What advice/questions do you ladies have to offer about my appointment? I would like to see a gyno or a fertility specialist, but I have a feeling based upon my last encounter with this doctor that he may not want to give me a referral. He is not allowed to do that, right?