Just need opinions

So long story been with my so for 14 years I wasn’t very overweight when we got together just had our youngest a year ago and when I gave birth I weighed 275 lbs the biggest I have ever been very quickly got back down to where I was before I got pregnant which was about 245 well recently started changing my lifestyle of eating and fitness and in the last 6 weeks have lost 17 lbs my goal is not to be super skinny but to just be comfortable and feel good about myself well he tells me the other day that if I keep losing weight he might not be attracted to me anymore just seeing if anyone has had to deal with this because I am doing this for myself and I’m not going to be unhealthy for him thanks for listening!!!