Leaking everywhere

C
I am so grateful for being able to breastfeed but ever since I started pumping, which has only been twice so far, it has increased my production too much. Baby can't keep up and neither can I, because I really don't need a huge supply at home. I wake up in the middle of the night, soaked through two breast pads, shit soaked, and baby can't drain the breast. I'm afraid to pump again to make it worse. 
Is there a way to pump without increasing supply?