How would you feel if he said that?

How would you feel if your partner said "Oh well there are plenty of other fishes in the sea"

We got on the topic because he said something like " would you break up with me" and I said "oh yeah (sarcasticly) which you could tell I was joking. And then he responded with that. He knew what he said was wrong because he kept apolgising and trying to make up for it but obviously I was hurt & he knows we've had issues about looking at other women in the past.