Antihistamine and TTC

I'm 31 and my husband and I have been TTC our first child for 7 months now, and are obviously, having some difficulty. I have terrible allergies and take Zyrtec daily, with a benadryl at night as needed. I recently read that taking an antihistamine can cause issues with conception and won't allow the fertilized egg to implant in the uterus. I have had several doctor visits to make sure that the supplements and medications I've been taking are safe, however, not once did anyone mention that my allergy meds could be the culprit. Any thoughts or suggestions?