How can you slow down milk production?


So the first few days after my milk came in, my now two week old daughter always refused my right breast so I just used my left. At this point, my right breast was insanely engorged and rock hard and my left nipple had started to crack and bleed. So I decided to try pumping and surprisingly I was also able to pump the right unused breast and I guess I got too excited and "over pumped". The next day I look in the mirror and my right breast is a deflated A cup and my left is a painfully engorged C. Since then (this was a week and a half ago) I've been trying to breastfeed with my right more to increase milk production but my breast doesn't seem any more full or any less deflated. My left breast on the other hand is constantly engorged and heavy and I often wake up with a soaked bra and breast pad, and puddle next to me in bed. I have to pump for a couple of minutes before feeding with the left otherwise the flow is too much and too fast for my daughter and she spits it all right back up, while she'll fall asleep on the right breast and is never satisfied because there's barely anything (I tried pumping that breast twice since it "deflated" and almost nothing came out).

How do I increase my milk supply in my right breast and decrease in my left?