Why is it that I always feel like shit after talking to him?

I have been with my boyfriend for 4 years. It always feels like when I talk on the phone with him, I feel like crying. I just called him to ask if he was on his way home since I stupidly left my keys inside my house so I could go to his. He said he’s still working. Understandable. I’m telling him how I have had a bad morning. (As to telling him what has happened to me). He suggests I workout while I wait. And when I say I shouldn’t since I have to work and can’t shower. He said then just wait. And then he says well I’m working. I have to let you go..

Am I been to sensitive to wanting to cry over this? It’s not the first time that I feel like crying after talking to him..