Has anyone ever had the SO take their keys and hide them from you?

Boyfriend works overnights and I try to cuddle or at least be in the same room while he’s sleeping. I got up to make dinner and he asks where I’m going. So I tell him I’m leaving- jokingly.

I go to the bathroom and then get dinner going.

Fast forward to him leaving for work and he gives me my keys stating that he hid them from me so I couldn’t leave and that it’s supposed to be funny.

I probably was in the wrong for just saying I was up and going to go- although I should be able to come and go as I damn well please. How or what would you do?? Thank you in advance for your input.