Struggling to leave baby...

I'm 3 weeks postpartum and having a really hard time leaving my baby. I know I need to get out of the house for my mental health but I just can't leave him, it makes me incredibly anxious and I start crying. I had a csection and I cannot lift him in his carrier, so bringing him along isnt really an option but even if it were that makes me incredibly anxious too. So I end up spending my days mostly laying in bed with him and I can feel myself just getting more and more isolated. I am even struggling with leaving him with my husband... Everytime he does a feeding or I ask him to watch the baby while I shower or something I have this overwhelming feeling of guilt for not being with my baby. Is this normal hormonal stuff? Anyone else struggling with something similar?