Ttc baby number two at 40

Im currently trying to concieve baby number two at 40 yrs old. My son is 4 yrs old. I've been off birth control pills for two years. We weren't technically trying but not preventing. My periods are every 28 days and just started taking ovulation strips and shows I'm ovulating. It only took 3 months to get pregnant with my son at 35. I only had sex 3 times that ovulation week. Do you think maybe I'm not having sex at exactly the right time?Maybe over thinking it?? I'm nervous because of my age. Should I go to the doctor?