New onset headaches in pregnancy?

I'm 31 weeks and my doctor was querying preeclampsia. I'm usually quite normal or even low for my bp. A couple weeks ago the readings started to trend high. They're not consistent, but certainly higher for me. Since then I've had headaches daily and the past couple days they've become more frequent. I was told to go into the hospital if they don't go away with medication. Last night it took two hours for it to go away completely, but it did go away. But first thing this morning I woke up and it's back again. They're increasing in frequency and duration. I don't know if I should go in or not because at what point is it just stress or a regular headache?