Arriving late to a date

He was 30 mins late on our first date. Then 40+ mins late on the second so I left and texted him to go home. He apologized and said he felt awful and would make it up to me. However I feel so annoyed and tired of putting up with it. I literally stood there waiting like an idiot. Am I being too harsh?