Feel like odds r stacked against us

We r ttc baby #2 & seems like every time fertile week comes around there is very little chances of baby dancing. Son is either sick, not sleeping very well or wanders in at all hours of the night, it is doing my head in! Not to mention ttc over 6 mths in over 35 club. Feels like not meant to have a baby brother or sister for our nearly 3 yr old son. Anyone in similar position or been there before & got pregant any advice would be good. Thanks in advance, good luck & baby dust to u all.