My baby doesn't breastfeed much every two hours

He is one week old, sometimes he would stay 10 minutes breastfeeding but sometimes he isn't hungry at all and would just suck a few times and then get back to sleep... Is it normal ? I try to give him the breast every 2 to 3 hours. I have to wake him up then.