Does age really matter anymore?

Does age matter when you connect with someone? My fiancé is 28 years older than I am but it doesn't make me love him any less. People stare at us when we go out and I just stare back. I'm pregnant with our second child and nobody seems to be excited like we are. It hurts my feelings because every baby should be celebrated and I'm not getting that from anyone but him. It's just crazy some people stare and others tell us how beautiful our child is. To me he's my soulmate and I wouldn't trade him but others think I should.