do you think I'm over reacting?

I have a rare day off work, boyfriend works completely opposite shifts to me. I rarely see him apart from one day a week because I start work at 8am, he finishes at 8am then is in bed when I come in from work at 5pm and gets up at 10pm then goes straight to work.. 
So on this rare day off, my boyfriend decided to hang around work for an hour instead of coming home. He told me he was leaving so would see me soon, then I text cuz he wasn't home yet and he replied oh I haven't left, taking to someone.  He knew I was meeting up with my friend at 10am so had to leave at 9.30, but because he hung around work for so long after I didn't see him. 
This really upset me, because I wanted a cup of tea with him, i told him I was looking forward to seeing him after work even if it would just be for a cuppa before I went out.  Obviously this didn't happen, he won't tell me who he was 'talking' to for over an hour and thinks it's no big deal., 
I'm really upset. Do you think I'm over reacting?