I’m bad for my baby…

I just had my first baby. He’s the light of my life, but my bf thinks I’m bad for him already. He had a traumatic birth and was in the hospital for 5 days. I wasn’t able to hold him a lot while he was there. So now that we are home, I have a hard time putting him down. I know I need to put him down, I just struggle to. My bf says I’m going to ruin him. That I’ll make him a Velcro child and that my bf will hate me for it. I joked that my bf doesn’t ever want to spend time with me and no one does, but at least my baby does. To that my bf said “well, that’s only because he doesn’t have a choice.” My bf says that me being insecure and thinking badly of myself will also ruin my son. So will me having a toxic mother. From the way I see it, he just thinks I’m a shitty mom all around and that my son would be better off without me. Maybe he’s right. Idk anymore. I’ve been trying to be a good mom. Being a new mom is hard though.