AITA/Need to vent

Current 13+5 with baby #2. We have an 11 month old son. Last week was really hard on my hubby as he had a friend/co-worker pass suddenly from undiagnosed advanced pancreatic cancer. Hubs struggles with depression anyway so he was pretty down. Friday night he went to hang out with friends which is fine with me, he does it almost every weekend. Saturday I waited for him to wake up to go finish our Christmas shopping but he opted out so I took baby and we were gone maybe 3 hours. Get home, things are good. He starts pitching baby names and inevitably gets silly. Then he says "we should just go with how he was conceived and name him Pity Sex." Obviously a joke. I joked back and said that I wasn't sure when baby was conceived. I had to back pedal because he took me seriously that I give him pity sex. I told him what I meant was that sometimes I'm not in the mood when he initiates but I still reciprocate. Thought it was okay. I go take a nap with baby and he leaves to go to game night (which that morning I said I would go, we'd take separate cars so I could come back if baby needed to) without telling me. I don't see him until Sunday morning. He'd moved to sleep in the recliner because his back has been hurting. I got up, was feeding baby. He gets up and doesn't say a word to me. Gets dressed, come by and tells me to go take baby after church to have lunch with his mom (previously planned) and that he was going to stay home to clean and get some house stuff done. Then he leaves super early for church without offering to help get baby ready so I could get dressed or anything. We end up late to church. I'm pissed at him by this point for just leaving and because it was really starting to feel like he didn't even want to be in the same room as me or spend any time with us as a family. Fast forward, his mom drives down instead and we have lunch. He doesn't eat. He's very standoffish all afternoon. His mom leaves, I ask if he's okay. He brings up the pity sex thing again, says it really hurt his feelings. Again I tried to clarify that I might not always be in the mood but I still want to feel close to him. I had to leave the room and went and cried. Just everything from the cold shoulder, to avoiding me and our son, and the idea that he would think that I would give him pity sex really hurt. That's something I think people at bars do to pathetic people, not a married couple. I didn't try to talk to him again last night. Focused on baby. He was still cold this morning.

Am I the asshole here? I'm really not sure. I felt I was trying to be honest. Maybe I shouldn't have been.