Should I let it go ?

So me and my bf have been dating for only a little bit now but, when we are around people (his ex) and others. He like teases her. But like we do that too. And I know he likes me but it bothers me. And the only reason they stopped dating was people stared gossiping about stuff about them. And I know exes can be friends and all but the way they act around each other bothers me. So like should I be bothered by it or not?