Oversupply of milk

Mi

My baby is almost 2 weeks old and the first week I had to do pump and give bottles the first week Because he was having a hard time breastfeeding, the past week I've been able to breastfeed him about 90% of the time. Ever since I've been mainly breastfeeding I feel so engorged and am producing more milk then he can eat. So I've had to pump along with breastfeeding but feel like pumping as well as breastfeeding is only going to increase my supply. Any advice on how to balance the two things ?