So random question

My husband and I have been married for almost a year now and together for 2. (Short, I know)

Well, we have had our share of ups and downs, well we don’t get into each other’s phones, we trust each other, but recently, I noticed that it says he is following my location (I have an iPhone and I was looking on my find friends app) and I know 100% I didn’t allow to share my location with him. I mean, if he asked me I would have been okay with it, but where he got into my phone and allowed my phone to share my location with him kind of bugs me... am I wrong to be upset?