how do you walk away from someone you love?

How do you walk away from someone you love to make yourself happy and do what you want. I'm currently in a relationship with my bf who was previously married and already has 3 kids. He's 29 and im 22. He's already been down the whole marriage path and experienced that and experienced having is first child with his wife and i will never get that with him because he wants completely different things in life. I love him so much and i enjoy being with him and i want to make him happy but he wants to have a 3 way relationship with his ex and i know that i won't ever get what she has and i want it with him but it's not an option and idk what to do. I don't want to walk away from him but i also want to make me happy and do what I've always planned in life. Any suggestions or advice