Age difference

So my boyfriend and I have been dating for a few months now, after being friends for years, and we’ve noticed a lot of offhand slightly insulting comments and some odd looks. I’m 14, he’s about to turn 16, and he’s almost 2 years older than we. He’s a freshman, I’m in 8th grade. We were in the same homeschool co-op class for two years, and became good friends, best friends, and then the beginning of this year we got together. He got held back a year so he should be a sophomore. People are giving us odd looks because he’s older than me. What are your thoughts on this?