Time to give up?

I know this not going to be as bad as others iv read trying for 10/15 years but iv been with my husband for 4 years basically tried from work go ( I was 36 and he was 35) we been married for 3 years and went to gp after no success with first 6/8 month went to fertility clinic and everything seemed great his and mine kept trying and waiting to get the dye test done, I did her a really bad kidney infection from one of my internal ultrasounds done at clinic and was hospitalised for a 2weeks so was not allowed to do the dye test this was April finally managed to get done in September and since then iv had constant overy pain on right side both badly blocked but right was giving a chance well we have been doing everything right and then this mint ago I went to check overy pain they can’t see anything and checked for ovulation and said they look for anything over 20 week after ovulation and mine was 43 so said if I tried then would be good chance well pregnant test is neg period due tomorrow so unlikely now that I am pregnant I’m just looking at my husband sleeping and thinking why keep doing this to us I am 40 next week and think because I waited so long to done someone, met him when my daughter was 13 and was single the day her dad walked away never wanted her to feel I never out her first, that I missed my chance and that maybe I should give up 4 years is not long I know but man why doesn’t it get easier how can an egg and sperm not know what it’s doing. I want to go to turkey to do IVF and we ment to be having a meeting with them on 28 of march but husband doesn’t seem keen on spending that money I’m just not sure maybe it’s time to say your to old

Now and missed my chance?

Sorry for long post just don’t have anyone else to really talk to about this.

Thanks