Help me please I have a huge crush and it's driving me nuts

So I have this huge crush on a guy he's 22 and I'm 26 we've been talking on and off since August but I never wanted to hang out cause I was content with being alone and bettering myself... Now that we've hung out I'm all into him and when we hang out he's incredibly sweet and we have a lot in common. But when we text he responds like once every 5 hours and it drives me nuts... He says he likes me but doesn't want to rush into anything. What am I supposed to make of this? Should I just leave him alone or keep trying? I'm getting major anxiety and loosing sleep over this.