Going with someone older

My boyfriends 10 years older than me and when I'm with him I can honestly say I totally forget the age difference but recently Iv had people saying stuff and it's really starting to get to me and I dunno what to do I know I should ignore but I can tell it's getting to him too ☹