newborn eating help

Idie
My LO is 3 weeks old, she went from eating 2.5oz every 3-4hrs for the first week and a half to eating about 3.5-4oz(doctor said if she can eat that much to give it to her at every feeding because that's what her body needs.) 
The past few days she has slowly been eating less and getting back down to that 2.5-3oz every feeding. Should I be concerned that she is eating less or could she have just been going through a growth spirt and needed a little more those couple of days? She only ate the 3.5-4oz for like 3 days and then gradually ate a little less after that. (Calling her ped in the morning either way)