Trust?

Kaila
My boyfriend and I started as a tinder hookup and casual fwb while he was constantly away for school. I dated other people while he was away, we were never official until he moved back last summer briefly before going back to school. We just get along really well, we're super compatible! He's graduated now and after surviving the trials of a long distance relationship, we've now been dating 5 months (2 months he's actually lived here). I keep learning things about his past that are making my anxiety and already complicated trust issues really hinder this relationship. He's told me he's cheated on girlfriends in the past, he's got a reputation for being the party's shittiest drunk, plus he's still friends with so many exes, one of them even lives in his best friend's house where he parties and crashes at all the time. He has never ever given me a reason not to trust him, he's been mostly open with me about the things he's done, but I can't help but think why I would be any different? I love him, but lately I just can't let my insecurities slide and let myself trust him the way I want to. I'm so scared of him making an idiot out of me for trusting him. I asked to look through his phone to give me some peace of mind (I have never had a desire to do this to anyone in my life) and even though I found nothing, he was really upset by how little I trusted him in that moment, I feel like I'm ruining this relationship with my inability to love the person he is without despising the person he used to be. How can I strengthen this relationship and build trust between us? Specifically on my end?