ADD Meds and TTC...HELP😣😣😣

My DH and I have been TTC baby #3 for about 4 months now. In early 2012 I was diagnosed with ADD and began taking Vyvanse. Once we started TTC I stopped taking them, with the thought that I would get pregnant quickly (as I did with my other 2), but that hasn't been the case. I'm also in my 30s now. 😁 Anyway, I have had to start taking it again because my ability to focus and stay attentive in my job was really becoming stressful and extremely difficult. I plan on stopping as soon as I find out I'm pregnant, which is why I have a stash of 50 HCG tests, but I'm still worried. The doctor says it "should" be fine since I will only be a few weeks along, but I don't want to jeopardize my baby's health. What should I do? Has anyone been in this situation?