not wanting to hang out with him

He asked if I could come over tomorrow, and I really wasn't in the mood to really do anything at all tomorrow! I just wanted to stay around the house and relax and play some video games. Of course I would still text him on the phone, but I guess I just wanted some "me time" tomorrow?

He thinks something is wrong with our relationship now just because I didn't want to do anything tomorrow... I kept trying to reassure him that it's not because he did anything, or that there's something wrong, but that I just wanted to simply stay at home alone and not do anything. Is it wrong for me to not want to do anything with him? This is the first time it's happened really, and I didn't think it would be a big deal, but I feel like I did something wrong.