Should I be embarrassed?? Kinda long but need advice

*keep in mind we are in our mid-twenties.

My boyfriend plays a pick up game every Friday and Saturday. This guy always goes and sits and talks to everyone. He is known to have far fetched stories. Usually, I can handle him because he's never really to to bad, just talkative. But last Saturday, he was insufferable. Being mean and talking badly about people who were playing the game, saying that they were weird and to not talk to them (he's older). He started being racist to an Asian woman who was working there. He was making it obvious he was unhappy with her work. I was getting pissed off. Now, when they get changed, the women and men change in the same dressing room (which has always slightly bothered me but I've never said anything). He started talking about how he could see this girls nipples and was telling her to put on a sweater and how nice they were. Then he decided to tell me that my boyfriend was nicknamed "grabber" or something like that because he would slap and grab another girls butt. Which made me really upset. He had details, like their conversation, where then my boyfriend slapped her butt, she said "hey that's mine", my boyfriend shrugged and grabbed it. I didn't want to ask any questions because I didn't want to get more upset. So I didn't know if this happened before me or during our relationship. So after a few more annoying comments from this guy (who I can't get away from because I'm watching the game) the game is over and I'm just done. It's obvious that something's wrong. When we get in the car I tell my boyfriend what he said. Even if it was before me, I would have liked to know if he did that only because before this story I had a weird feeling about her and him from the way the played. The game isn't very physical and they would always push each other around or push each other into the boards. They would pin each other against the board and try to get the ball. And they would try to sit next to each other. He said he never did that and that this guy was lying. So my boyfriend called the guy who gets everything together and told him the situation. This guy offered to tell me it wasn't true. My boyfriend also asked for the number of the guy who said it. Because he's my boyfriend and this guy often says things that are far-fetched, I believed my boyfriend. I'm not sure why he would blatantly lie like that when I could easily talk to my boyfriend. Anyway, now I'm embarrassed to go watch the games because a couple people know that I was concerned. Should I be?