How to break up with someone?

I’m 21. I’ve been with this guy for 4 years and don’t get me wrong I love him but honestly I think I can do better and I don’t see me marrying him. We’ve been through alot and there are things that I can’t forget. I’m young and I feel like I’m missing out, but I care about him and dread hurting him. His mom and sister left him and his dad kicked him out. He lives with his auntie, and thing is he says I’m all he has and when I leave him he’s gonna have this view of everybody he loves is always gonna leave him. But I can’t stay with him either. Help?