Told to stop prenatal vitamins....

I have a really interesting question, I have never heard of this so I wanted to see if this has happened to anyone else. I'm 10 weeks pregnant and they did my blood test and everything and I'm all good and healthy but I lost 6 pounds in 4 weeks. My doctor told me to stop taking my prenatal vitamins saying that it was making me nauseous so it's making me lose wait. I have never ever heard of someone being told to stop there vitamins. So my question is has anyone else been told to stop taking them? I really don't know how I feel about not taking them. I know my doctor knows best then me but I just really don't know.