Have to be referred to Specialist?!

Maddie
Just wondering if you see a fertility specialist or RE.. Did you have to be referred from your gynecologist or could you just make your own appointment? 
Considering seeing a specialist this coming year but not sure how to tell my gyno.