I don’t want to lose him...

So the other night my friends & I decided to get high. They had done it a lot & that was only my second time. Anyways we got high & went back up to the boys room (their was 2 guys, brothers, & me & my best friend) always the older brother & my best friend had been doing sexual stuff for awhile & I think they were planning to again but the ended up passing out. Me & the younger brother used to date but had decided to be friends. But when we went back to their room me & the younger brother ended up cuddling, he had then put his hand on my inner thigh. & well we started making out. I thought it was fine until he tried to get in my pants. I pushed his hand away. So went back to making out & then I don’t know things went so fast & we were both so out of it. He told me to be quiet & pulled my pants down & by now I was like okay sure. Then he pulled his pants down & we were going to have sex I know we were. But he’s a virgin & I’m not. So he had trouble finding my hole...& then he struggled so much that he gave up. He’s so embarrassed & honestly I would be too. But he got up pulled his pants up & so I go up and got my pants back up. I told him to come back next to me & he did I kissed him again and he stopped me & said maybe we should go to bed...so I kissed him one more time & told him that it was okay & left. He hasn’t talked to me since & I’m scared that I just lost one of my closest friends...what should I do?

By the way I did text him the following day & he left me on read. He talked to my friend & told her that he’s just embarrassed which I get. But am I supposed to just wait for him to be ready to talk?