what do you think about this

Okay so me and the guy that I was talking to send me this emoji 😱 and I was only joking around with him and said that is the emoji you use when your pull out game is weak. 
And then he writes back to me and goes lmao you don't need to tell me about your past. I don't need to know that.
So I got a little offended because I felt like wow we usually joke around and you never mention my past or say anything rude like that. It did bother me what he said because my comment wasn't towards him. I was just saying in general what people use. idk how to take it. Does it sound fucked up or am I just over reacting ?