20 weeks. Weight gain question!

Kristin • Married 8/16/14💍 Pierce Jaxon: 6/5/16💙👶🏼 Logan Marie: 2/27/18 🎀👶🏼.
Hey guys. So my dr. Says I am gaining too much weight 😳 (the worst thing to hear from your dr. Lol). But he really didn't give me any advice other than "the holidays are over, stash the candy away". Well.... I wanna know how to keep the weight from rising too much or harming the baby (even though he said my weight isn't going to determine the baby's weight). At our ultrasound this past week baby boy was measuring 10 oz which was technically 4 days ahead of schedule. Should I be worried (I really don't want a huge baby!!) and what can I start doing exercise wise to help with my weight! 
Here is what I do now:
We already walk about 3 miles every other day. 
I am pretty active at work. And I eat okay (I know I could eat more veggies... Carbs and fruit just are my weakness this pregnancy). 
Thanks ladies. So embarrassed to be in trouble for my weight haha!!!