Anxiety meds vs not while ttc?

What is everyone opinion on not taken anxiety meds while TTC? Do you think the anxiety alone could affect not being able to conceive? How many would take the meds maybe until they got pregnant?