Help! Baby will only sleep on me

My son is 4 weeks old and will only sleep in me or husband. He has never slept in his basinet as he freaks out when laid down on his back. He hates being swaddled too.

I don't know what to do. Want to get him in his crib but he won't sleep. I've tried waiting till he's passed out or putting him in when he's about to fall asleep. I've even tried putting a blanket underneath him that smells like me and is warm .... we have a rocker/swing that he sometimes will sleep in for a nap but not much.

I'm so worried he's never going to sleep well. Plus I would like some sound sleep as sleeping with him on the couch doesn't allow me to deep sleep.

Help!