Foot surgery while pregnant?

So a little over a week ago I fell down my stairs. I'm 11 weeks pregnant , the baby is safe, but my foot on the other hand is a mess. I thought it was just a sprain so waited to get it looked at (5 days) turns out it is broken in 1 spot( my 5th metatarsal) and some shattered pieces on top. The 2 doctors that took a look at my xrays said I'll probably need surgery to fix it. But I do see a specialist tomorrow morning to get another opinion. I'm terrified of doing surgery while pregnant. Has anyone else had to do surgery like this while pregnant? I'm going to talk to my gyn if they do think I need surgery and see what his opinion is too.