Advice on weening off the pump?

Cady • Mama to 2 with 1 on the way!
My daughter is going to be 2 weeks on Tuesday and is primarily bottle fed. I wanted to exclusively breastfeed but she was born at 34 weeks and breastfeeding was too hard for her so she was losing weight trying to eat (she has a great latch and the perfect instincts for it, just not the energy). So since she was 3 days old, we've been pumping and bottle feeding, we tried to just supplement but after she started getting bottles she would refuse to nurse. Yesterday she showed interest in nursing again, so since yesterday afternoon we've just been breastfeeding. But I keep getting so engorged and have to cave and pump! Any way to make it easier to ween off pumping and just breastfeed?