Bf issues! Please help

Okay so yesterday my bf and I where hanging out with his guy friends and everything seemed fine. They where drinking and enjoying there weekend. When we went to drop his friends off, we passed a group of college girls on the side walk and he stated "man sometimes I wish I was single.." And he's friends cut him off, telling him I was in the car with him. He looked surprised and started stating he didn't know/forgot I was in the car. I need advice on what to do. We are going on 3 years and it's our second year in college together.  I just don't know what to think. I'm just torn.