Acceptable or Unacceptable for your man to glance at other women?

When I catch boyfriend glance at other woman and I call him out. He says it isn't the case. If I look aty surroundings when people pass(people watching)  but he thinks I'm looking at other men. When clearly I know the case but he says the same about himself. His actions definitely speak louder that he only desires me but this is the one topic issue we have in our relationship.