At what point..

I'm sure we've all been at a place in our relationship where we think to ourselves "is it time to give up?" I'm stuck between giving it my all and giving it all up. So at what point does one walk away? Not from a marriage, just from a long term relationship. Advice?