Anyone else freak out when baby has a lazy day?

Even though I've been told it's perfectly normal, I HATE when my son has low movement days. I know they have lazy days just like us or are sleeping because they're going through a growth spurt, but every time my son does it (which is probably every 2-3 weeks or so), it makes me super paranoid. I always think something is wrong.

Before I got pregnant, I had a doctor who was very harsh to me about my chances of conceiving and maintaining a pregnancy (I have PCOS and hypothyroidism) so even just getting pregnant was like a miracle to me. But every time I have the tiniest symptom now, I get so worried. Both I and baby have been perfectly healthy thus far and my current doctor assures me everything is fine, but it's so hard for me not to panic when he doesn't move for half the day!

Does anybody else tend to get freaked out when baby has a low movement day? What do you do to assure yourself baby is fine and to calm down?