Am I wrong for not wanting to take off work for our 5 year anniversary next month?

My husband and I will be married for five years next month on the 16th. The problem is that I just went back the work on September 2nd. I was a sahm for 3 years, but I’ve went back to working as an instructional assistant for an elementary school. We’re doing virtual learning from home for now. Our day ends by 11:00am because it’s hard to keep preschool aged children online for long. Right now I’m working like 3 hour days. Our anniversary is on a Friday. I figured that I could work the 3 hours and then we would have the rest of the day to ourselves to celebrate. He’s not okay with that. He wants me to take the whole day off. I can understand why, but with me just starting not even a month ago, I don’t think it would be a good idea to take off already. With things being online it can get kinda crazy and confusing, and I’m just trying to avoid all of that. I also don’t want it to look bad on my end with my job. However, at the same time I don’t want to upset my husband. I will only be working for 3 hours in the morning.

What would you do?

Edit: Nothing is planned. He keeps asking me what do I want to do that day. It’s been kinda hard to actually plan something because of the pandemic. We don’t even have a babysitter yet.