Should I take offense

Hi so I’ve been trying to work on this baby weight but it’s been so hard with work and time so I mean I’ve tried dieting and eating right but I don’t have time for exercise ANYWAY so I was at the store and I wanted a shirt and my boyfriend grabs it and says LARGE?? Gee I remember when you were a medium and he said it out loud in front of the cashier and now I just over all feel like crap because every time I shop I have to remind myself that I no longer fit into the small clothes I used to wear and I always wish I fit back into it. Then for someone to throw that in my face and say “I was only kidding” like Huh am I wrong?? Should I take offense