Weight gain during pregnancy.

I'm 27 weeks with my first and I've gained 4lbs now. (I'm now 198lbs.) I've always been heavier but have never had any weight-related health issues. Today my doctor lectured me about the weight I've gained so far and told me that I need to slow down. He was pretty rude about it and talked about how our society accepts obesity too much. I was so embarrassed, especially because I've been trying my best this pregnancy to keep a slow weight gain! Do you think this is a lot to have gained by this far along?