Zinc question

So I've been taking a prenatal vitamin for years now as recommended by my obgyn. Last year I was diagnosed with ulcerative colitis and started losing my hair. My nutritionist friend thought it might be because I wasn't absorbing enough zinc from my vitamin. I started adding a zinc supplement with my vitamin and my hair stopped falling out. I stopped taking the zinc 2 months ago. My hubby has a terrible cold and without thinking...I took a zinc (50mg) to try to keep from catching it. I'm only 6 dpo...do you think I just ruined any chances of getting pregnant? What was I thinking!?!