How can I make them understand.

Hi, I'm a single mom, to a 1yr old. As of now my ex doesn't really watch our daughter, Since he had a leg injury, which I understand. Even then, He would watch her every other weekend, For a few hours. Anyway, my problem is, How can I date, when my whole dedication is my job and daughter. I talk to this guy, who seemed to understand at first my circumstance. But now some what gets irritated when I tell him that I can't just up and leave my daughter, he does want me to bring her along. But I some times need a little me time, and tell him no. How can I make him understand that I'm a mother first. I don't feel right leaving my daughter with my mom since she's the one who babysit's during the week. I don't really have the help from anyone else..