Advice?

My baby is 3 weeks old and WILL NOT sleep unless he's latched on to me. I've tried swaddling, paci, rocking and all. As soon as I take him off he wakes up fussing. He's been doing this all day today and it's currently almost 11 pm and is still doing it. Is this normal? Lol any advice?