Oversupply

I think I might be dealing with an oversupply of milk. My boobs leak constantly and are so engorged they hurt. I massage them/do warm compresses/take hot showers to relieve engorgement and my milk literally squirts everywhere when I massage them. They drip when Im just sitting here and its making it hard for my little one to latch on. Anyone else having this issue? I was pumping the first few days because he wasnt getting enough milk but since my milk came in I stopped pumping completely. What do you do to relieve your oversupply?