Are we too young? Honest opinions please

So we're 5 months pregnant, I'm 24 he's 26, we'll be 25 and 27 when baby is here, we've been together 7 years, have our own house, good well-paid careers, have had 7 years of having holidays alone and are not ones for partying. Our family and friends are over the moon for us, however work colleagues and patients at my work keep commenting that we're way to young....what's your opinions?