Nighttime Feeds Are Driving Me Crazy

I cannot for the life of me figure out if I am waking my baby to feed before he’s ready. He’s 6 weeks old.

He’s a loud sleeper and he lays there grunting and rolling around, but never cries. So about every 3 hours I get him up to feed him and of course he starts crying when I unswaddle him.

How do I know when to feed him? He’s supposed to be able to sleep 4-6 hours at night by now (in theory, I understand all babies are different) but I can’t tell if I’m getting him up when he wants to or if I’m misreading things and getting him up before he asks. But he never cries. Just lays there grunting.