When is it time to give up

My husband and I have been ttc for 2 Years now and I'm starting to give up completely. We had an ectopic last July and I am now with only my left FT and I get pains all the time...sometimes sex doesn't even feel all that great and I just know something isn't right at all. I've been looked at and my doctor says my left FT looks fine and my ovaries and uterus look fine and gave us 6 more months of trying before we did more testing...well 6 months is officially up and Idk if I even want to figure it out...we can't afford ivf in any way and we both have 6 year Olds to other ppl...I feel like I should can the idea of having another kid...I just don't think it's in the cards...has anyone had anything like this happen to them and when is it time to give up?