leaving a man with mental disorder

My boyfriend suffers from depression, schizophrenia and bipolar disorders. We've been together for 4 years, and he started to get worse after a year.  At first, I accepted that dating this man means accepting him fully for who he is. As much as I love him, his illness has gotten to the point where I've had to skip school, change my schedule at work to give him time, dealt with manic episodes and his hurtful words. He also refuses to get help and take his meds. I care for him deeply, but I am so unhappy and don't even know myself anymore. Is it right to walk away, or is it selfish? Any tips on how I can help him?