Can doctors take away a diagnoses?

I was diagnosed with a mental illness but my new psychiatrist only saw me for about two minutes and said he didn't think it was correct. 
He is notorious for doing this, does not have a good reputation..
So can he re diagnose me, or take the diagnoses away? I am just curious.