Xanax when ttc?

My husband and I are ttc and we have a trip in a few days that requires flying. I take Xanax every time I fly for anxiety. I've tried not taking it and it's always resulted with a serious panic attack. I should note that I do not take this regularly, only for flying which we do roughly twice a year. Our trip is going to be in that 2 week waiting period so I won't know if I'm pregnant. Should I just take the Xanax and avoid the panic attack or should I not risk it? How harmful is it if I am pregnant?