I'm terrible at this

Should I invite this guy to spend time with me or no? Long story short he was a new guy at my gym on a day pass, we started talking and he ended up spending two hours with me patiently teaching me a technique, joking and laughing and showing me the ropes on it. I had no clue he had already been there for three hours before that until he told me, and i felt so awful for taking up his time. We left when the gym closed and he took down my number and gave me the biggest hug ever. A part of me says to just leave him alone because he was just being nice and probably doesn't want to be bothered again. I have no one else to ask and i need opinions!