Why doesn't he reply to my text?

I recently started dating this guy. And it's been wonderful every time and he's so sweet when we meet irl. But it always takes a few hours, sometimes a couple of days, for him to reply to my texts. It bothers me so much, I get mixed feelings from him. He seem to like me when we meet but then he acts like he want to get rid of me.
And I'm starting to fall for this guy. I've been texting 4 texts now in a week with no reply, I think I'm going mad. Should I call him? Even if this ends now I'd like to know what the hell happend. And why he's playing with my feelings. What do you think I should do? And what do I do to get my mind on other thoughts?