broke up and he's mad I went to McDonalds with my kids father

We broke up a week ago because he won't even let me mention my kids fathers name, let alone talk to him.
Last night I picked my kids up at 9 after flying in with their father. 
I forgot my wallet so their Dad offered to buy us drive through McDonald's.
Then I dropped him back off at the terminal.
My now ex asked my about how it went. Then he asked why I was being defensive.
(I knew he would get mad)
He pushed the issue and then I told him.
He got incredibly mad at me, said this is why he doesn't want to be with me and stormed out.
I'm left feeling like a bad person! Did I really do anything wrong? 
How do I explain to my ex boyfriend that What he is doing is not right.
He makes me feel like I cheated! 
I understand being jealous, I know it's hard but to take it out on me like this?