Baby fussing at breast at night.

So my four week old is breastfeeding like a champ but when I don't feel full or a little bit engorged, usually at nighttime, he will fight and fuss and cry at the breast. He'll pop off and relatch and just be frustrated which makes me frustrated too. And I am at a loss on what to do or why he's doing it. I don't know if it's positioning which since he's gotten bigger I have a tough time finding comfortable breastfeeding positions for the both of us or if I'm not filling him up enough or if its just because babies do that, I don't know. Any advice or ideas on why he's doing this? Anyway I can fix it? Do you think it's my supply decreasing? I try to keep him on the breast for a long time to keep my supply up but I don't know. Thanks!