Weird breast issue

I've had this off and on for years. It comes and goes depending if I'm on the pill or not. It's there off the pill but not there on the pill. I've been off the pill for a little over a year now. About a week ago I noticed it again, it's only in my right breast, I feel like bumps and lumps, it's sore if I push it, I've had a little bit of leaking from my nipple. I'm not pregnant. Well I got my period today and the bumps and lumps have shrunk and barely any soreness now. What could it be?