Looking at SO's phone?

Do you think it's unreasonable to go through your significant others phone? Every time I look at my bf's phone I find something semi-incriminating and he gets mad! I tell him if he had nothing to hide he shouldn't be mad! Is this crazy?