Sex only when he's in the mood

My husband thinks just because he's working he should lounge around most evenings and comes to me when he's in the mood. But if I'm up for it he's too busy playing games or is tired. I get angry because he turns me down very often and prioritizes more often his "me" time rather than "we". I think he's being very selfish. I've mentioned this to him but he doesn't respond, I don't know what to do?