Pregnancy with ovarian cysts

I want kids, but am enjoying traveling & being the two of us. I had a durmoid cyst on my ovary 2 years ago, and in the removal process they took my left ovary & tube.  The doctors just discovered a very small cyst on my right ovary & there is a slight chance it could be a durmoid.
​The doc has given me approval to try for kids now, but my husband is concerned it could affect the baby or me. What are others thoughts on this? Anyone been here?
​Should we jump into the baby making before we are ready just because I want to have kids before another surgery?