I must be doing something wrong

Nunu
My son is 11 weeks. I literally spend my entire day with him. I don't get a chance to do anything.. When he sleeps or naps he has to lye on my chest, the moment I put him down he wakes up. When he's awake he can only stay in his swing for ten mins then he gets fussy. What am I doing wrong?? Or is this normal?