Feeling bad....

Ive been in a relationship im not very happy in. Every time I go to break up with him he cries and i feel so bad..... his family is sick right now so i feel like i have to stick around. I feel like im not living my life to my full potential. He doesnt have goals and I have been with him for 3 years and he hasnt grown at all as a person. I just know this isn't what I want i just need some advice and im selfish if i leave ....?