Adderall and pregnancy

Jennifer
I am on 15mg adderall xr 1 time a day. My husband and I are TTC! I've read some people take a small amount and everything was ok with baby but primary care doc says to stop taking it as soon as i find out. I only take my meds on work days