He took back his key ?!

So my boyfriend and I have been together for almost 6 months, and overall things seem great despite the distance. We communicate regularly and make efforts to see each other. However, the unexpectedly happens when he asked for his apartment key back when we met up recently, and I’ve had it for two months now after he given it to me. But now he claims he never intended for me to have it, that it was suppose to be for only when I’m in the city. This, along with him sharing his location right after me giving him the key back after he previously saying he wouldn't because he would feel as if he’s being monitored, all of this has left me feeling uneasy. I'm not sure if I'm overreacting, but it feels strange for him to take back something insignificant like a spare key if everything is truly going well between us.