age difference

I'm 20 my SO is 25. Is that a bad age difference? I really never cared about age, thought it doesn't matter. But lately when we talk about it with his friends/new friends I meet they give him shit for being with someone "that young". It gets on my nerves because we've been together for 3 years and I'm really mature for my age. Everyone forgets I'm only 20, they think I'm like 40 lol. But having so many people say that now, I'm starting to think is it that bad? I wouldn't end things over it or anything like that. I'm just curious if it's really that big of an age difference that it's like wrong?