Grumpy baby

...is anyone else's baby grumpy pretty much the entire day? He is 2 weeks old and when he is awake, he is crying. He is fed and burped and changed and held constantly and he just cries all the time. There are maybe 2 hours out of 24 hours that he is awake and quiet at the same time. I feel like I'm doing something wrong