Am I in the wrong?

I drove an hour to see my boyfriend this morning before he has to go to work at four, I even brought breakfast for us both. After he’s eaten he lays down (he naps quite frequently) and we cuddle but within minutes he’s dead asleep and I can’t fall asleep (how it typically is). I ask him if he wants to go for a run, which he had said the previous night he needed to, but he said no. I left the room and occupied myself with my computer. I waited an hour before going back in and asking if he was ever gonna wake up, he said no so I got my shoes on and started to leave. He said once to take my shoes off, but I explained to him that I wanted to see him, not watch him sleep. So I left and I feel like an asshole for leaving but also I drove all that way just for him to sleep the whole time. Am I in the wrong?