I'm in a interracial relationship and we get dirty looks?

I'm a white woman dating a black man and we get dirty looks like we are doing something wrong I can't believe in 2015 race is still a issue with some people? We were out to dinner and I over heard a black women saying that one that a white girl took away a black man from a black women? I don't understand why can't we just see people in love and be happy for them?