Should I let it go or confront him?

Ok so tomorrow is my birthday and I was under the impression that my man and I will spend it together (we don't live together) well when I asked him today he said that it probably won't work out tomorrow. I'm obviously disappointed and upset but I was wondering if I should ask him why it won't work and demand that he gives me a legit reason as to why he doesn't have time for me on my birthday which is also his day off. Or should I just tell him ok don't worry bout it and let it go? I don't want us to argue but I also want him to know that I was looking forward to spending my birthday with him again just like last year and that now my feelings are a little hurt. Suggestions on what to do? Also he gets very stubborn so I feel like if I'd ask him why he can't tomorrow he'd get mad and say that it should be enough of an answer when he said that it won't work. Ugh