Waiting for the right time?

Sarah
My husband and I have talked about babies for months, but it never seems like quite the right time. There's always a new job or a new house or a new something! I feel like there's never a 100% perfect time, and we should just go for it. But he wants to wait. 
​I'm just worried that it's going to take forever once we do start trying.  I've been trying NOT to have a baby for so long that I just feel like karma is going to get me when I do want one! I know plenty of people conceive after birth control with no issues - I'm just nervous!
​What's your opinion? Wait for the right time? Or make it the right time when baby decides?