3 weeks old

Candace
My son has been crying all day. If I hold him he's fine. I can't put him down, not even for a minute. I feed him, bathed him, put on baby soothing sounds. He will not lay down on his own. I'm going NUTS!!!! 
Should I bring him to a hospital? Maybe there's something wrong. 
I've been sick the last 3 days with a stuffy nose and now a sore throat.  I'm breastfeeding him. Maybe he's catching something ughhh!