Advice for a sad gal

It’s my boyfriends 23rd birthday and I gifted him a weekend trip away, the trip has cost me well over £500, not that price matters. When he opened the card, he said “why didn’t you choose me something instead”. My heart sank and I couldn’t find any words. I didn’t buy him clothing because he wears designer stuff that I can’t afford as a student. I feel so guilty and upset for choosing something he didn’t like, what do I do? He’s since said he likes it and that gifts don’t matter anyway. I just feel so bad.