'Stop trying to conceive'....

I know people keep saying to 'stop trying' and it'll happen, but my problem is if we stop trying, it won't happen...bc DH and I have only been having sex during my fertile time, when I say we should. Otherwise we don't for the whole rest of the month.