Anniversary/Valentine's Day

Today is of course Valentine's Day but it's also my anniversary with my boyfriend. A few weeks ago he asked me what I wanted and then said he was joking he already had my gift.  Well this morning I gave him his gift and he didn't give me anything so I figured he was waiting until later.  He told me a little while ago that he didn't get me anything because he didn't have the money, even though he knew in advance about today and all it meant to me.  Am I wrong to be upset?