Baby growing ahead of schedule!

I am 24weeks and 2days pregnant. Yesterday at an OB check, they said that he was measuring around 26 weeks. So they ordered an ultrasound for Monday. 
At my 18th week appointment, he was an inch bigger than he "should be" and my ultrasound tech didn't act like it was a big deal. But now I'm getting worried. 
I know I'm 24 weeks based on my last period and my very first ultrasound.
Is this bad that he's growing so fast? I'm starting to get anxious and nervous 😥. I feel like he is fine but my doctor's order of an ultrasound, concerned me. 
I am 5'7 and my son's dad is also 5'7. But my side of the family is very tall. My dad/brothers/uncles are all around 6'1-6'5. Maybe he's getting his height from my side. But is this cause for concern?