psychiatrist wants to stop my bipolar meds?

I was originally diagnosed with bipolar last august. I saw a psychiatrist at my college who agreed with the diagnoses and changed my meds and the dosage. Since i’m home I have started seeing a new psychiatrist. We have only had 3 sessions but he doesn’t agree with the bipolar diagnoses and is chalking everything up to anxiety and wants me to stop my bipolar meds to “have the real me work through these anxiety issues.” In the 3 sessions we have had together we really haven’t talked about the traumatic events in my life or my emotional problems. All we have literally done is make my family tree and touched on my daddy issues. The psychiatrist i saw at my college knew about everything that happened, how I felt and how I deal with things in my first visit with her. I have a meeting with my regular doctor on monday and he told me he didn’t want her increasing my meds and if anything he wants me to stop them. Is it worth continuing meeting with this psychiatrist? I do believe the meds i’m taking have helped stabilize my moods so I really don’t want them to change.