losing my mind!!

Br
My son is 3 weeks and has been so fussy all the time lately. It's starting to get to me. He will be fine and then out of no where start crying and if I move him he will stop for a couple seconds then start again. Until I move him again. He will have a fresh diaper and just had food and do this. Just this morning he was done eating. And wouldn't take his paci but would be happy with chewing on the empty bottle....and fuss after I moved him in many different directions. Then put him on my shoulder and he was out instantly....is he just going through a growth spurt? Colicky? Just being a turd? I'm going crazy and I can't get anything done when he is like this.