Relationship advice?

Recently my boyfriend and I have gone off to separate colleges just an hour away from each other. Since he has been there he hasn’t really talked to me much and just doesn’t seem to be interested in me anymore. When I brought it up to him he said it hurt his feelings that I thought this. We had planned for me to drive to him the day of my birthday, which happens to be Labor Day weekend. Today he told me not to come because he was going to be busier than expected. Later after that he said he won’t be able to talk to me the next couple days because he needs a mental break. I have cried the last week over what to do because I still care about him so much but I don’t feel like he has been putting in the same amount of effort into our relationship as I have. What should I do?