Harassment ...?

I work as a nanny, I took the child to the doctors and the doctor made inappropriate comments and made me extremely uncomfortable.

He started by taking my own phone out of my hands to talk to the mother of the child (I was calling her to ask a question) he just took my phone and pressed the "non-speaker" button like if he owned it. Then he told the child to open her mouth but because she did not want to he commented "Let's see if nanny opens her mouth" but he said it clearly in a dirty way. Not in a funny, innocent way... it was disgusting. He didn't do anything else specifically but I just didn't like how he spoke to me ... I decided to tell my boss (the mother) she laughed and she just said "well i don't know what to say" so I felt awkward for sharing it with her.. maybe i shouldn't have but I felt like she should know...