Help, Should I divorce?

We are married for 4 years now, we sometimes have arguments and fights and usually it is because my husband hides his phone from me - he says it is because he does not want me to “misunderstand” something on his phone . Recently I found out that he had a dating app installed couple months after we got married but I did not find any messages on it before I confronted him. When I confronted him he said he did it because he felt lost and not feeling really ready, that he was just stupid but he swears he never did anything to cheat on me, he never texted or met anyone on it and that he loves me very very much. There has been some similar incidents like these where I doubted in his loyalty but since I had no proof I kind of moved on from it, this time it feels different since I know he had an intention of doing the “deed”. I am so confused I don’t know what to do at this point, it was a long time ago but it still feels like he is not being honest about it, how do I move on from this? How do you start all over again? I love him deeply and it hurts me so much because I know I have to leave. He says it kills him because he sees how much he hurt me and he wishes he never done it, begs for another chance and promises he would never ever do this kind of a thing again