Mom guilt..

I want to stop breastfeeding because I think it's making me hang on to extra calories and making hard for me to loose weight. I'm back to my prepregnancy weight but I started off heavy at 240.. by the time I gave birth I was at 300lbs. I'm 5'9 and although being back to 240 is ok, I still am not happy with my body and I think breastfeeding makes it hard to loose any more weight. I'm doing weight watchers and I've lost 5lbs in 2weeks, but idk I'm feeling depressed, defeated.. the minute I decided I'm going to stop breastfeeding, my baby feeds and he is so skinking cute smiling at me and looking up and I right away feel so guilty that I want to stop so I can be thin.. shame on me.. 😩 any advice?