Confused :(

My husband always complains that we never get to have play time because our kids.. so I decided that once the kids go to sleep that I'd try to wake him up and maybe have a little fun? But when I did he got like super upset/annoyed and told me to leave him alone? And now I'm just confused? I already feel like I've neglected him enough and then when I try ? He's rude about it.. so what am I supposed to do?? It's a crappy feeling...