Please listen :(

Ok so i had sex with this guy back in March and Idk if i wanted it or not like I did but I didnt. Anyways last week his friends asked me about it and i was like yeah we fucked and then they started making fun of me so i was like " i didn't really want it tho" and then they implyed that he raped me so on Friday me and him got into a huge argument about it because apparently people think he raped me because it got out. Anyway now we don't talk and it makes me sad. I apologized to him and told him i felt terrible. I don't want to lose him. What do I do?