Umm commenting on weight?

I'm about 13 weeks and have gained 4 lbs and when I was at a family gathering people kept commenting about how I should watch my weight while im pregnant and not to eat too much. 
I told them I'd gained 4 pounds and the doctor said that was healthy and a family friend (older man) said "well you don't want to gain ANOTHER pound" when I went to eat cake😁 what the?? Is this normal? I'm not overweight, I'm pretty petite. I was just so taken aback by people feeling the freedom t talk about my weight while pregnant? Anyone else experience this?