Discouraged

Does anyone feel guilt for not always wanting sex late at night after a hard day? My husband usually on the weekends will play games and say he wants to "spend time with me"... I've been trying to be more flexible and forgiving as he is trying to get better. But the other day I said no to sex simply for the fact that I had been puking, not in the mood, and while I was sick he wasn't there to take care of me... tbh I was hurt. When I passed his offer, he got upset. Saying I thought he wasn't attractive etc... when I have never once put him down. I told him one line and one line only "if I wasn't fully happy with you I wouldn't be your wife, your girlfriend, your best friend."... idk I guess I'm just tired of feeling like it's something I owe to him... advise?