Guys are a little to preoccupied with "size"

I hooked up with this guy once and once we started he was like "do you like it" and I was like "yeah"(it was all right but we weren't really going for that long). But then he was like "am I big?" And he wasn't! He was like average size. I've had bigger but it wasn't a bad thing. So I was like "uh huh". THEN the condom fell off and I was like "dude what the hell"? And he asked if he could go bare , and I was like "nah man. Bye bye." After he left I saw the condom wrapper, and he was trying to use like XL condoms for his average af dick!! I feel like I should have told him, but I didn't want to hurt his feelings. I just am annoyed because there was nothing wrong with it. But him wanting to be "big" ruined the whole thing!! Why do guys care so much? Obviously there are expections , but I feel like most of the time...it's not how big you are it's how you use it 😝