Phone password

So my man and broke up because his ex-girlfriend's texting and wanted to get high with him. So you started to fight with me and left I was gone for about a week. I was done and he showed back up. He said he wanted to make it work he went down there for a getting help and I've been clean since I've been taking the medicine he's supposed to be taking and helping out here and there. Now he's looking for a full-time job. I have his password to his phone but he wants my password to my phone. I don't feel that he deserves my password. Don't get me wrong I don't do anything sneaky or wrong. I don't text my ex or anyone I shouldn't be texting. And he knows that. Am I in the wrong for not wanting to give him my password?