Am I being selfish?

I'm currently considering leaving my fiance and his kids for my dreams.

So long story short: we're bad influence on eachother, we overspend it's not even funny.. We have like 4 loans and a car loan and we're renting a place suitable for 5 people.. So that's a lot of money going out the door..

Honestly I kind of blame him for having so much trouble. I didn't want all those loans.. They are in my name only because he couldn't get any because he was still on the loan bill for the house his ex lived in and I wouldn't have taken on these loans without him.

And I'm mentally just really broken.. I'm 26 and I feel like I have no perspective in life.

I feel like I spend not a single cent to myself all of our money seems to go to him and his kids. And I even feel guilty for even thinking this because I don't want to deny his kids anything.

The thing is that with him, it doesn't seem like we can save at all, I'll never own a house, I'll never be able to make my dreams come true. Unless I end this.

In my ideal world I would live small and cheap with my dog and save a lot and in a few years I would move abroad.

Unfortunately because of the loans and the living cost of living with 3 kids it's impossible for us to save (sufficiently) to even pay the loans.. Let alone to save for a house..

I am lost...

Update: it would already cost me years to recover from all these loans, before I can even start to save money even if I leave today. But alone you can save your ass off.. But you don't want to deny kids a decent meal..