Is Stealthing Rape?

Stealthing is the practice of removing a condom without the knowledge or consent of the other partner when it was agreed upon to have only protected sex.

In most places this is considered rape because it is having sex in a manner that was not consented to.

However, classifying it as rape has proven to be very controversial, with many mostly men saying that if sex itself is agreed to, breaking agreements about the details of sex isn't rape, only a very stupid and disrespectful thing to do, and some (mostly) women saying that classifying this as rape makes their more forceful and violent rapes mean less.

I think that sex and the details of sex in any form that wasn't consented to is rape. What do you think?