5 weeks postpartum

I am 5 weeks postpartum. Exclusively breastfeeding my son and just started pumping to build a supply. I lost 18 lbs in the hospital when I had my son and I haven’t lost a thing since I’ve been home. In fact I keep fluctuating up & down 2 lbs. I know this may sound silly but it’s a shot to the heart when everyone says “oh the weight will just melt off” and I’m struggling over here. Any mamas out there experience this, have suggestions or can shed some light? Thanks in advance!!