Remove fibroids before ivf or leave them???

Hi all, I have two fibroids and I'm scared I'll have to delay my cycle to have them removed. Has anyone experienced this? 
Best wishes & baby dust to all