Nipple pain

I've been breastfeeding and pumping to try to build up my supply. LO is estimated to only be getting about an ounce from me at most at each feeding and therefore is currently on formula supplement as well. He is 2 weeks old and I've been Breastfeeding since day one but all of a sudden as of yesterday I am having severe nipple pain to the point where I don't even want to wear a bra because anything touching hurts. Has this happened to anyone else and will or go away? Also any solutions for pain relief?