Is it wrong

Ok so say you and your SO live together, and he/she leaves the house in the middle of the night while you are sleeping, is it wrong to feel some type of way or to be mad about that in any way shape or form. Last night went to bed about 830pm, and my bf was in bed with me, I happened to wake up at 945pm and he wasn't there, I went to the living room and he wasn't there either, and I looked at the door and his keys were gone, I was pissed, went back to sleep but idk what time he got back. Idk if I should bring it up or not. This is the second time he has done this.