Fussy baby, worse during early morning

My 3 week old daughter is EBF. It seems like most nights around 3-4 am she gets extremely fussy. She'll go to sleep then wake up not long after and start crying and acting like she's hungry, but when I try feeding her she pops off and cries before relatching. I try giving her a pacifier and she sucks on it for a minute before fussing again and spitting it out. I've tried burping her, changing her diaper, swaddling, rocking, just laying her down, nothing seems to work and it goes on until about 6am when she finally stays asleep longer than 30 minutes. She's fine the rest of the day and some nights this doesn't happen. Could this possibly be something I'm eating or is this normal for most babies? My son never did this so I'm at a loss.