Anyone else experience this or am I crazy?

So I've noticed that my breasts become swollen and hurt the most at night. During the day I fit into my regular bra and barely any pain. I wake up maybe 4 to 5 times during the night to use the bathroom and it's always so painful because they are so sore and I feel like they blow up! Does this happen to anyone else? I'm wondering if it being cold in our room has something to do with it. We live in the northeast and it gets pretty chilly at night and we haven't put the heat on yet. So at night we are always under the covers.