So confused

so this guy has liked me for a while but I was not feeling it, I was always with someone else. finally I felt as if I can give in a little and started showing more interest. he's been iffy and not showing as much but SOMETIMES he will invite me to dinner and we've hung out. well we hooked up (not sex) recently. and he is so standoffish. he won't text me back sometimes and doesn't ask to hangout much. recently he invited me to go to his house (tonight) and I asked him when he was off work. NO REPLY. what the hell is going on!