How to end it

How do i end a realtionship with some that has a kid that has grown up with you around their whole life. We have been together for over 5 years and his kid has known me since less than a year old. He asked me to marry him and i said yes at that time because we were good together eveything was great we had a relationship but now things have changed and no matter how much i try talking to him he doesn't see it. He acts like a child trying to bug me constantly because to him its funny. I want to leave but my fear is leaving his kid that i have a very strong bond with. His kid comes to me for everything before him. I dont want his kid to feel abandoned by me but i know i need to move on. How do i do this with a 5 year old kid involved that is very smart nd has gone through this with other people but while she was younger(she was 3 when it happened the first time not with her father but someone else close to her)? I really need advice on this