Sore nipples

I'm 16 weeks and whenever I'm not wearing a bra and my nipple touches fabric it feels like I'm being stabbed! Anyone else have this problem? The worst is when I get out of the shower and use a towel, just touching it hurts!