When to start TTC?

Did you ever have the "moment" of wanting to try to have a baby? I know nobody says there is a perfect time, but was it being settled in a house? A certain amount of money saved up? I love babies.. I am a nanny and an aspiring photographer and doula. My Hubby and I are in our 30s (not that age matters that much) I only have one ovary and the doc has sort of told me "to hurry up if I want to try"