Trying to declutter my wardobe

So I’ve tried the Marie Condo method of only keeping what sparks joy but I’ve run into a problem. Over the past few years I’ve gained a tremendous amount of weight, because of this none of my clothes that actually fit me spark joy. I hate the way t shirts look on me, my joggers I wear to work etc. I’m just having a very hard time loving my wardrobe. I have a ton of clothes set aside that are currently too small and I hope to fit back into some day but is it really worth keeping them or is it doing more damage than good?