Fibroids?

Anyone have any issues with fibroids during pregnancy? Went for our first ultrasound today and they found a big fibroid (way bigger than the baby!) and said it shouldn't cause too many issues but feeds off hormones so it will continue to grow. Both my aunt and mom had them but after their pregnancies and both ended in emergency hysterectomies. I can't help feeling a little scared. Anyone have any advice or have been through this?