Mother’s/Fathers Day-for spouse

So some co workers were asking me if my husband had plans for me this Sunday. I replied, no and he shouldn’t anyway, I’m not his mother. And they asked if I do anything for him on Father’s Day and I told them no because hes not my dad. I was given looks of disgust and as if I’m a cold ass hole person. Im my opinión, I have our wedding anniversary, birthdays and Christmas to do something special for each other. Does this really make me seem rude and ungrateful of my husband?