what would you think if your doctor wore sweatpants and polo tshirt to work daily?

Would you think anything of it? Would you doubt his professional opinion? Would you even see him or would you go elsewhere or ask to see another doctor? Does it matter what he wears to work? Would you still think the same if it were a woman?