Large fibroid on the right side of my uterus

Anybody else have any experiences with having a fibroid during pregnancy? I knew I had a fibroid before I got pregnant but it has grown significantly since becoming pregnant. It is now 11 cm and is considered large. I had my 20 week ultrasound last week and everything is fine except the fibroid is distorting the lower half of my uterus. The fibroid is not in my uterus, it is next to it. I was told at my fertility clinic that it would not affect pregnancy but now because it is so big my doctor is concerned that I might have to have a C-section because of its location. A vaginal birth may not be possible. Anybody else have anything similar and went along with having a normal pregnancy and vaginal birth? I’m reading things online and it sounds like most women have no issues but they are concerned because of the size and location of mine. They’re sending my scan over to a specialist but I just wanted to check on here to see if you ladies have any insight for me.