bad friend? awkward situation.

When I got out of high school I started babysitting for this lady. I was starting college and it was good to get some income with out having to have a real job. It was great, I love her children dearly, I babysat them for over 2 years. Now during these 2 years she got divorced, and started to date around. Lots of guys in and out. I was in a relationship during this time but we broke up. So long story short she moved to a different state for a fresh start. One of the guys she dated during the time between her divorce and her moving started texting me. We were friends too because he was around alot. Long story short weve been on a few dates and I dont know how to feel about it. Is this wrong or no?