What should I say?

So I had my daughter 8 months ago and I have gained about 50 lbs since having her. Before I was thicker and now I am more on the obese side. I struggle with it but i try to stay positive and love my body. I'm not with my daughters father and have been seeing a guy (FWB type thing) that I had dated before getting with my daughters father. He just told me that he doesn't find me very attractive since seeing me after having my daughter and kind of hints a lot at me losing weight and throws around stuff like "I have seen plenty of girls drop the baby weight right after having baby that is no excuse" which is obviously annoying but I try to ignore it because he is clearly ignorant. Anyways, he invited me over, which makes no sense because if he finds me unattractive why is he asking me to come over? I'm just conflicted on what to do i guess because I don't know how to feel about all of this. 🥺