Need some advice :(

Tyler
So my boyfriend jokes around all the time. Today we were talking about getting a gym membership together. When I asked him where he would like to get a memebership at he said at the university because they have pool tables and a place to play basketball and he heard that there are hot women there. This comment instantly hurt me. I have low self esteem about myself because of past relationships where I was always put down and cheated on. He knows this but still makes these horrible comments. Am I just being to insecure or is it really horrible that he hurts my feelings?