He's "tired of dealing" with me

Up until last night I thought our relationship was doing really well, but yesterday evening we got in a huge fight. He gets off work at 4 in the evening, and when he wasn't home by 5, I started to worry so I called and texted him, no answer. I never got a response, I knew he got off work on time because I texted his coworker. I sat at home for hours, no response. Then, 6 hours later, he walks in the door at 10 at night. I asked where he was and he goes "I don't have to tell you". I asked if he was cheating and he goes "no. You should trust me" so I said if it's nothing bad, why can't I know why you disappeared and didn't let me know. Eventually he said he was out with a military buddy, but I feel like if it was that innocent, he wouldn't have tried to keep it a secret. He blamed me, saying he's tired of seeing me everyday and that's why he didn't respond for 6 hours. I don't know what to do. I'm at a breaking point, I've been crying all night.