Boy Trouble: Is it Time to Give Him the Boot?

So, my SO and I have been seeing each other for almost 5 months. I knew going into he was a workaholic and very involved with his job. He's an entrepreneur and very addicted to success. However, I feel like he makes me take second place to his job alot. Let's just say he has cancelled more than three dates we've had...in fact I don't think we've ever been on a real one. Ive forgiven him for this in the past yet idk how much more of it I can take. I told him it makes me sad when he disregards me. However he always says how much he needs and wants me which makes it pretty confusing because always has an excuse for why he has to cancel plans. I feel like all we do is have sex. Texting is a game of how long I can go without talking to him first before he caves and texts me first. I always feel really bad about myself when I text him first, especially if I get no reply then after. And I mean I like the sex but I wish we could do more stuff and actually talk more. I haven't seen him in a while so, upon my return to college from the summer, we were supposed to reconnect today. However, no text from him all day. I didn't text him because I'm tired of feeling like I have to initiate everything. So, we did not see each other and I'm sad cause I was looking forward to it. Should I let him go or just be patient with him and his busy life?