Nosy neighbors & property ownership, what to do?

We moved in recently to a house that my parents purchased for my husband & I. We are paying them the mortgage payments and will eventually own the house. They figured it would be advantageous for both sides.

However, our neighbors looked up the property info for our house and are now snobbing us, saying things like “I knew you couldn’t afford that house yourself” and other comments behind our backs. What would you say? I’m pretty pissed.