How to deal with a partner that battles from severe anxiety?

My partner and I have been together for a year now and I’ve noticed that he battles from very bad anxiety and he’s been honest about that himself. However, it’s now getting to a point where it sometimes has an effect on me even though I try my hardest for it not to. For example, today we went on a date to a Mexican restaurant. When we first arrived, it wasn’t a lot of people there. However, around the time we were close to leaving more people started to come in. Since COVID exists his anxiety started to go crazy and he was ready to go. I understood but we had to wait for my order to be replaced. Less than 10 minutes later we finally left. I tried to get him in a better mood..put music on..started being goofy and dancing in the car..I even tried holding his hand and making him laugh but then he moved his hand away from me and said he didn’t want to hold my hand. I felt just a weird tension in that moment and just remained quiet. It makes me sad because I wanted our date night to end off in better spirits. Then once we got home we barely talked and he went to bed. This isn’t the first time his anxiety has kinda brought my mood down and I don’t even know how to express that without him becoming defensive or feeling like I’m not caring about his anxiety. As I’m typing, he woke up and laid down in the living room.. I understand COVID is serious especially since my mother doesn’t have the greatest health conditions ..but why did the night have to go that far? Can someone help me understand..?