First birthday idea

I know it's too early but I want to ask your advice! My son is a Halloween baby and want to start to plan his first birthday! I don't know which theme would be better. I am a big Harry Potter fan and always wanted to have a Harry Potter themed party. But my son will be too young to remember if I have his first birthday party to be Harry Potter themed? Should I wait until he can actually enjoy? I want other kids we invite to wear costumes also since it will be Halloween. Would it be easier for them to have costumes on if it's just normal Halloween theme? Should I just do normal Halloween themed? But I want it to be also special because it's his first one. I can't decide 😩