So I need some advice

When my baby was born she was taking formula due to her having jaundice and having a tongue tie which was hard to latch. Well I was double pumping and feeding her with a bottle but i have stop now and I’m doing all breast session. My problem is that when I feed her on one side i leak from the other one due to her not feeding on that side. Well I have to pump out that side so I won’t leak over my nursing pad. I want to ask if any of u were pumping and then stop and if ur milk regulated. My MIL told me if I hold onto the milk it can cause my supply to slow down or dry up my milk. Please help my LO is going to be 3 months on Wednesday.