11 wk old only naps in carrier

Carissa • new momma to ethan leonidas
new mom, need help! 
my 15.5 lb, 11 week old will only nap in the ergo carrier with me wearing him. i get no breaks during the day and can't even nap with him if i wanted to. if i even try to set him down, drowsy or asleep, he immediately loses it. ive tried just patting and soothing him, picking him up putting him back down but he just cries as hard as he can until we are both crying and i finally just put him back in the carrier where he doesnt even sleep well because i am trying to get things done.
am i trying to make this transition too early? do i start with one nap a day of this traumatizing transition? or do i just go cold turkey with getting him to sleep in the carrier? or do i just wear him while he naps until he gets too big and my back gives out?
sincerely,
eternally bouncing