Saying no to sex

Do you ever say no to sex? If so do you feel bad or do it just for him? I'm worried my husband will be upset if I say no to him.