7weeks+4 with fibroid

I’m almost 8 weeks and I have a subserosal fibroid . B4 I got pregnant I asked a surgeon if I should remove it but they said if I was to get pregnant it would not bother the baby because it’s on top of the uterus it will only cause me some discomfort and pressure . Well I’m already feeling pressure and the baby is only as small as a bean . My fibroid was 6.7 cm b4 the pregnancy then when I went in to check I was 6 weeks the fibroid was then 8 cm. I was already prepared for the pressure just didn’t expect it so early . I also heard the first trimester is when fibroids grow the most hopefully it’ll stop after that .