Slightly concerned..?

So, my boyfriend and I have been not trying/not preventing for a little over two years now and I'm a little worried that nothing has happened. I know we aren't technically trying (I only track my period and the times we have sex..and migraines, but that's unrelated lol) but most months we're definitely active when I'm supposedly ovulating. Should I be concerned? I feel like asking a doctor is pointless because we aren't charting or really making an effort other than when nature beckons.