Reading to much into it?

So I have went on five or six dates with this guy and he’s been really sweet and showing how much he is interested. Well last night we went to dinner and we were waiting in truck for text for table. He seemed to be a little more like he didn’t want to be there (even though he brought me chocolate) and started saying things like “if we work out.” I was like telling myself don’t read into it. Then we set down and he proceeds to tell me that his ex unblocked and messaged him. He seemed irritated and kept saying “it’s always when your moving on.” I was wondering to anyone else would this be a concern. Just seemed odd that the night after she messaged him he is seeming a bit negative and a little less engaged in what we are doing. Don’t want to get caught up in a situation someone isn’t over ex is all? But I mean how would you talk to someone about it? Should I just give another date and see? I should also mention the text have become less between the two of us and then he says this last night and that’s why I maybe felt ehh.