Referral to infertility doctor

So today was my first appointment with a new gyn, I have felt that my previous doctor was blowing me off with my concerns so I moved on. I was there for an hour and he pretty much told me that he is concerned that I will need assistance getting pregnant. I have endometriosis, with constant pain. I have been trying to track my ovulation and have never gotten a positive. My cycles are 43-45 days apart. Longest story short, by the end of the visit he referred me to a specialist and wants me to get all of these tests done. How did you guys take this news? I’m terrified and angry. Angry at all the people who can conceive so easily and angry at the doctor who didn’t listen to me when I told him I knew something was off.