Age Difference

Se
Does age difference really matter that much? I'm 18 and my boyfriend is 27, but ever since we announced that we're together we've been getting nothing but negative comments from people. I knew ahead of time that we would be judged, but WOW am I ever surprised by the amount of judgment we've been receiving. I'm still going to college in 2 weeks and my boyfriend is fully supportive of me chasing my dreams and doing a long distance relationship. 
How did everyone else handle this situation if you were ever in it? 
And is age really that big of a deal?