Chronic headaches and pregnant

For the last 3 weeks I've been getting anywhere from mild to severe headaches and migraines. At my first prenatal visit I mentioned this and at the time it had only been a couple days of them and doc advised to try Tylenol. Since then I've been having them more often and more severe. Has anyone else experienced this? Should I be concerned? Should I call my doc again? This is my first pregnancy.