Conceiving with ovarian cysts

Had anyone had a really hard time conceiving with cysts? My doctor said that when the time comes that I want to start a family it could be difficult. And I just want to be prepared for a struggle if that's the way it will be.