Metformin & First Trimester

Hey guys. I did Femara to conceive and I'm currently 4 weeks and 5 days. I have PCOS and have been on Metformin for years and years. I am seeing a high risk OBGYN because I also have thyroid disease and a rare adrenal disorder. Anyways, she told me because I'm at very high risk for gestational diabetes to stay on the metformin until 12 weeks. Any other woman would be told to stop as soon as u get a BFP. Anyone else told this as well? I'm just a little scared that it will hurt the baby since usually you are spost to stop taking it