Work Bullies

I keep getting told what I can do and what I can't do by my coworkers. I drink a small amount of coffee because my doctor doesn't feel it's a risk. I drink less than a cup. I'm trying to figure out how to politely tell them that their advice is heard but I don't intend on following their no caffeine diet suggested to me. I just don't want to be rude. I'm really annoyed that they harass me everyday when they see me with coffee. I don't want to retaliate in the way I feel I'm being treated. Anyone else go through similar situations? Or have advice?