TTC bloodwork results

Ttc and went for bloodwork. Low thyroid (knew this already), high prolactin, normal progesterone. Dr immediately told me to consult an Reproductive Endocrinologist. Isn't that a bit pre-mature? I feel like they just threw me IVF as my only solution! Shouldn't I just get the hypothyroidism in check and everything else will fall into place?