Anyone else been pregnant with bipolar/depression?

I was diagnosed with depression as a young child and bipolar depression 3 years and have taken medication on and off. I also have a rare neurological nerve disease, Trigeminal Neuralgia. With that said, I just spoke to my Dr.'s nurse as this will be my first appt (I'm 5wks with #2), I told her I have stopped taking my medication and she insists I stay on it and talk to my dr who has been prescribing it to me. Now, I do NOT want to be medicated in any fashion during pregnancy and I stopped with my 1st pregnancy as well and everything was great, I had 0 "episodes". I should mention I am in no way a threat to myself on or off my medication. Also, my bipolar depression medication interacts with my neuro disease and causes even more pain. I'm kind of dreading this first appt as I know my new Dr. is going to think I'm a hot mess.