Stuck in a rut

My sex drive has always been lower than average and my husband and I just had a baby almost 4 months ago and my sex drive is low again. I got off of birth control because of the fact and now being post partum I am still off of it. My husband and I argue about sex constantly. Every day, multiple times a day to the point where I am disgusted by sex. I think a lot of the reason I don't feel the need or want to is because he has been super lazy, and sleepy apparently. He will stay up until 3 and 4 am and play Xbox and won't wake up until 11 am and I am the only one working currently. He lost his job due to breaking the code of conduct but doesn't remember what he did to get fired. So I am stressed over bills, money, breastfeeding is still super difficult since I breastfeed and pump and work 12 hour shifts. My son has been going through the 4 month sleep regression and waking every hour at night to feed. My husband has not gotten up 1 time since he's been born. My husband will spend all day on Xbox or God knows what else that when I come home our house is still dirty, if novt dirtier because all he did was eat and sit on the couch all day. I am just very frustrated and overwhelmed at the moment and he is living it up it seems and is demanding sex and I just do not want to have sex when I can't even sit stressing. Am I wrong? As a wife do I have to have sex with him even if I am unhappy with him and his actions?