Bf left his phone at my house annd..

Last night my bf came drunk to my house & I had to find him a ride home cause he wanted to stay with me but my parents wouldn’t have allowed it. He left me his phone since he could lose it & told me to drop it off in the afternoon today. I didn’t snoop through his things, I’m not like that. Today he kept getting a call from his friend (guy) & I didn’t answer cause obviously it’s not me who he wanted to talk to lol & I figured after a few missed calls he’d think my bf was busy or something & stop calling or leave a message. He texted my bf “let’s go to Galveston (beach), I got some girls ready” I hadn’t seen it until he sent the second message saying “or are you scared of your girl”

Now, I was mad & told him “Ivan left his phone at my house last night, I’ll let him know once I return it”

I’m so mad!! I’m not going to tell him to cut anyone off because again, I’m not like that. How can I come about this without seeming controlling?