Does size really matter?

Hi ladies so I recently started dating a guy we met at a bar I automatically felt physical attraction towards him, still getting to know him we’re basically in the talking “phase” still. We’ve been dating for a couple months now i like him a lot he’s very respectful but he does have a couple red flags..

One night we decided to meet at a hotel I knew things would get physical.. we ended up being intimate & it was a total fail.. he couldn’t stay hard & it lasted for a very long time & to top it off he’s VERY small like pinky size small. I vented to my friends about this & they all tell me it’s not worth keeping him & to not waste my time.. I’m new to the dating world so I’m pretty sad over this.. am I wrong for cutting him off because of this?