Scared to start TTC again after a break

I am really lucky to have my son but I’m struggling with trying to have a second baby to complete our family. It took us 8 months to conceive our son but we tried for 1.5 years without success for our second so we stopped trying. We’ve decided to try again but I’m just nervous for all the disappointment. It’s so frustrating idk why this is happening and I’m scared to death that something is wrong with me and I can’t have another baby. Anyone else going through something similar? It doesn’t help that I’m 32 and so desperately want another baby. I feel defeated like it happened once why can’t it happen again?? I haven’t gone to a fertility doctor but would like to it just feels like an overwhelming process and figure out where to even start