Leaving a Job Due to Anxiety?

Today I quit my second job due to stress, anxiety and depression. My husband fully supported the decision, and so do my co-workers at my main job. I don't need to work two jobs to keep us afloat, but I chose to because it was something I enjoyed. However, this job was very demanding and stressful. Between planning, supervising, and being a manager at my main job it just became too much. I'd have constant stress headaches, my heart condition worsened, and I became depressed for the first time in my life because I felt like a failure of a family member because I was almost never home to watch them grow. I was going to give a two weeks notice, but was nearly blown off by my supervisor. She thought I was using anxiety as an excuse, so I just quit. I admit that maybe I could have handled it differently, but how can someone be so rude about anxiety? Am I wrong for this, should I have stuck it out, or is what I did justified?