Gained 20 lbs in 23 weeks

Josie
Feeling a bit depressed after my last doctors appt. Looks like I gained almost 9 lbs in 4 weeks. Last month I was really trying to walk more and eat healthier (I definitely fell off the wagon a few times going out to eat). Doc had a pretty frank convo with me about gaining too much weight (she even gave me a chart of how much I should gain and how much I have gained). I was a big girl before I got pregnant so I've always been a little worried but doc hasn't said anything until now.  Feeling really down on myself.   
Anyone else experienced this? I just want my baby to be happy and healthy (as we all do) but I'm scared I've done damage that can't be undone. 
I asked for a referral to a nutritionist and will completely cut out any dining out and keep up with my walking but I don't know what more can be done.