TTC...kinda

So my partner and I decided for me to go off birth control in June. I'm 26, he's 39. We were best friends before we got together, we've been living together for 2 years and decided we really wanted a child and we are in a good place to have one (emotionally, financially, etc.). We both thought it would happen naturally super quick. Well it's been 5 months now, I've been tracking ovulation, period, etc. But, we haven't been having sex on all my fertile days, we just didn't think we would have to "try". We decided this month that we are going to really try. It's so scary, because what if we're really trying now and I can't get pregnant? I had a fertility check up and everything checked out. Has anyone else been in this place? Any suggestions before this undertaking? I don't want to stress our AMAZING relationship more than it needs to be. I know to some of you ladies this might sound silly because of all the hardship you've gone through TTC (btw so many amazing stories on here) but it's been weighing on both of our minds, like if we admit we're really trying it might end up that we're really failing. :/