Sexual Harassment?!?!

I had an encounter with a male employee 2 days ago and I'm still offended but not sure if I have a sexual harassment case on my hands or not?!?! This co-worker overheard a private convo between my manager and I and assumed. He said to me "You're not expecting AGAIN are you?" I have 5 kids (which is none of his business) and if I were blessed with more I would be grateful. Most of my co-workers don't understand my decision to "have so many kids" but personally it's none of their business. So he then went on to say "Haven't we talked about this? I thought I already told you?" Meaning he already explained to me where babies come from and to stop more babies in the future. He has never discussed this with me until 2 days ago because, again, NONE of his business!! I'm still very offended 2 days later as he has become more hostile with me in his conversations. What should I do? Turn the other cheek because haters gonna hate? Or report it to my manager and fear the retaliation afterwards??