Advice..is this a red flag?

I have an 11 month old daughter that I don’t really let anyone meet and this man I met through my job recently. We haven’t talked much at all but he asked if he could see me tonight, well I live far from my family who baby sits my daughter while I work. And I kindly told him that I would’nt be able to meet up because I don’t have a sitter near. And he got very short worded with me...and I told him maybe I’ll try and find a sitter for next weekend and he just sent a thumbs up. And then I told him to be safe driving down to Tennessee tonight and he only sent a thumbs up

Needed to hear all of this, thanks ladies :))