Did I overreact?

I was with a group of friends including the guy I like last night. We decided to carpool somewhere and when one friend asked if I was gonna drive, the guy I like exclaimed" she can't drive". My night driving isn't so good and I got in an accident a few weeks back. I've been self conscious about driving since then and the guy I like was there during the accident. I told him I was annoyed and he was all," why get mad if it's true" am I wrong to be annoyed by this?