Valentines and love language problems

Me and my my giancee have always had problems with out live languages not bad ones but some and valentines is a problem too. We have been together for 5 years and his love language is physical touch but mine is gift giving which makes him think im materialistic and thats thats all I care about. I have tried to explain that its not about the gift itself but its the fact that he thought about me like he could literally give me a rock he found outside and tell me that it reminded him of me and I would be over the moon or a letter or a sticky note or a tiny chocolate. I really dont nag about it I started to get myself flowers from the grocery store when I go horecy shopping because he says its a waste of money buying flowers when all its honna do is die, and I buy my self what I want its not like Im using him to buy me things. I always put thought into getting him cool presents gor his birthday or valentines or christmas because I want to and thats how I show love but after 5 years it does get me sad that I put so much into suprising him and He dosent. He mentioned something about goong to dinner for valentines like a week before the 14th but hasnt brought it up anymore and I feel bad asking over and over so I was going to wait and see if he brought it up again. I jokingly asked what he was gonna get me for valentines and he got upset and said “oh well I guess I just wont pay next months mortage to get you something” and it made me so sad like last year he told me I dont have money sorry I cant get you anything. He had a really bad job last year, and I thought it would be nice to send him something anyway so I sent him a edible bouqet to his work and he loved it. Now he has a really really good paying job and Its not like Im asking him for a expensive thing its the thought that counts and he makes it seem like thats all I think about. I pay part of the bills too, work full time and take care of our three girls and I feel so unappreciated when he makes it aound like thats all I care about. I didnt even know what to tell him. Do I sohnd materialistic? Is he right?