Breast in pregnancy

Before I knew I was pregnant I picked up a nasty habit of smoking cigarettes and even though I don't smoke now is it possible that being a previous smoker slowed down the process of breast growth? I usually have bigger boobs at this stage with my body preparing for baby , but this time around it's very little growth. Did I possibly slow down production? I'm concerned. I'm 6months by the way.