3 week old crying it out at bedtime

My son is 3 weeks, close to 4 weeks old now. He’s staying up throughout the day for much longer periods of time than before and towards the end of the day he becomes very fussy and cries which I’m guessing is from being overtired. He constantly looks like he is fighting sleep. I will feed him, change him, hold him until he falls asleep and as soon as i put him down he wakes up screaming crying. This lasts from around 4-5 pm every day until 8-9 pm when i try to get him ready for bedtime. I also give him gripe water for his tummy just in case anything is bothering him and he’s on a hypoallergenic formula as well so it shouldn’t be medically related since he’s otherwise healthy.

Tonight I put him down in his rock and play and he cried for 5 minutes straight but then fell asleep. Is this okay? I feel awful letting hm cry but I can tell he’s so tired and every-time I pick him up he wakes up for at least another 30 minutes or so.

Anyone else dealt with this? Is he just overtired? Feeling clingy? What can I do to fix this?