Fibroids and Pregnant.

I recently went to the doctor for pain that am having over my vagina and the ultrasound results show that I have fibroids. The doctor said they're small so he's not concerned and they will shrink when I begin menopause but I am aways away from menopause and he knows that my husband and I are TTC our first child but he said to keep trying it will not affect the baby but my pregnancy will be very difficult.

Is there anyone that dealt with fibroids while pregnant? I really want a baby but am terrified of a "difficult" pregnancy.