medication question!

So before I found out I was pregnant with baby number 2 I was diagnosed with an anxiety disorder and was was given Ativan and propanonal to help with panic attacks. I was only taking a very low dosage about 3x a week. I am 6 weeks pregnant and stopped taking the medication as soon as I found out at 4 weeks. I called the doctor and he said that I could continue to take only as needed but I am nervous to take them still. Just wondering if anyone has been prescribed this during their pregnancy! Thanks:)