my ex rubbed my upper back and I was upset. did I overreact?

my ex goes to the same boxing gym as me. he used to emotionally and verbally abuse me when we were dating. aftwr a few years of being broken up, I tried to be cordial and forgave him but I would NEVER date him ever again. he tried talking to me a couple of times after a long while at the gym, and then the next time he rubbed me a briefly on my upper back to say hello. I was so angry because I don't want to be touched and so he was all smiling and I had a death look on my face and sounded monotone when he was asking how I was. mind you, I let no guys touch me except for my friends who are girls. [edit: in the past when I ended up getting a new boyfriend, he would try to talk to my boyfriend while we were at the gym together and when my bf wasnt there he would try to flirt with me and touch my leg briefly while he would pass by. I am single again]. when I had the death look on my face and was monotone, he was like taken off guard and then was pissed off because I was not okay with him rubbing my back. I never said DONT TOUCH ME but I tried to make sure he got the message by looking visibly upset. he did it so quick and then left so fast so I didn't have a chance to process what happened and went into survival mode and just reacted in the way that I did. was I overreacting? what would you do in this situation?