Breastfeeding weaning help!

I have been slowly weaning my 18month old from breastfeeding. It’s been 4 days since nursing and my boobs are kinda engorged. Not super painful yet.. just feels like a thousand knives stabbing me whenever my daughter bumps into it. Haha. My question is, will the milk that’s in there already dry up? Or will I need to pump it out? I don’t want my body to think it needs to make more, so I haven’t pumped at all. Maybe just enough to take the pressure off? What should I do?