How are your partners handling your weight gain?

Amanda
This is my second pregnancy.  I've always been a bit of a twig.  With my first pregnancy, I put on 60lbs and it took about 9-10 months postpartum to shed the weight. This weight fluctuation tends to be the norm in my family, as my sister, mom and aunts have all done the same. 
I'm now 21wks in and gave gained about 18 lbs.  A few days ago, my husband glanced and me and had a visible moment of slight disgust; I asked him what was the matter and he said, "oh nothing, I just have to get used to you being full figured again".  It caught me off guard but I didn't say anything at the time.   I mentioned casually to my husband over dinner last night that I'm hoping not to put on as much as last time, and he said, "yeah boo, I hope not; you got really big.  I mean you weighed more than I do." I'm now recalling comments he's made since I got pregnant about how I hopefully won't gain as much this time around since I'll be so busy chasing our toddler. 
I know that he's not being outright mean and that it's probably awkward--even unpleasant--to watch your wife slowly transform into a beluga.  During my last trimester of our first pregnancy he barely even touched me, and didn't show any interest until several months after our child was born.  I'm now overthinking it all, and I am not happy; I'm suddenly becoming a lot more critical of my body instead of just rolling with it.   
Any of you experience this? What'd you say, if anything?