Traveling while pregnant

Do you have to tell your OB if you're going to be traveling? Can they drop you as a patient or something if you don't tell them?