TTC but then Fibroids!

My doc just told me I have too many fibroids and 2 that are in the wrong place. Said it would be hard to implant and even if I implant, baby won't stick. She has referred me to an infertility endocrinologist because she says with my age, we don't have time to waste. Anyone here has any experience with Fibroids while TTC?