Annoying comments about skin color

Does anyone else have people who make comments about baby’s skin color in a regular basis? My father in law will constantly ask anytime we come over whether the baby is getting lighter or darker. I have to stop myself from responding and asking him why it matters.

I have darker skin for a Hispanic person and my husband is also Hispanic with darker skin so I’m not sure why my FIL is obsessed with her skin color. I grew up very insecure about my skin color and wishing I was lighter because of all the comments I heard about from family and friends. I used to get both positive and negative comments about it so I really just don’t want my daughter growing up hearing anything about her skin period. Is that an unrealistic expectation to have for family members? To not make comments about it?