IT'S OUR 4TH ANNIVERSARY AND IM EXCITED TO IT

Hi guys, next month is our anniversary. But he just told me that he won’t be available for 3 days because he’s invited to the wedding of his sibling's friend. I told him it’s a shame that it’s our anniversary, and he replied "Yeah, it’s a shame hahaha" and that’s it, I didn’t hear anything more from him. I thought he would say that he could skip the wedding to celebrate our anniversary, but nope, he quickly changed the topic to something else. He knows me, I’m not really into fancy dates or gifts; I’m happy with just spending time with him in simple ways. I don’t know if I should ask him to skip the wedding so we can have time for our anniversary, or just let him go because I don’t want him to think that I’m restricting him and not letting him do things like that.