Relationship Help

So I feel like I could be over reacting, so it might just be my anxiety. Because I don't feel anger, I feel sad and just anxious. So here's the BACK STORY: When my boyfriend and I started dating, I told him that I was super against coke (brought up because I found a bag in my apartment - thought it was my friends but he said it was his roommates and he picked it up off the table and out it in his pocket so I wouldn't see it). Not people that do it or anything, but having it in my personal life is not something I want. I have made that clear from the beginning when we were just FWB. After dating a couole months, I found out he did coke and that the bag was HIS coke that night (also taken that night). We went through a load of shit with that issue (I already had major trust issues, now I felt like I couldn't trust him either), but decided that we didn't want to be with anyone else. That the relationship we had is the one we wanted to fight for. (Before this, he had already told me his friends did coke - I don't care, I just don't want to be around them while they're doing it). Fast forward to 6 months, I really wanted to go to a concert (which he liked the band too), so we went. Beforehand, we discovered one of his friends (let's say his name is Phil) and his wife would be there. I knew that this Phil guy was a user and asked my boyfriend to ask him if he was going to be supplied that night and depending on the answer we'd either say hi and chill for a few minutes or actually chill the whole night. Come to the night, I thought he asked because he offered me a seat at their table and we STAYED. All of a sudden, Phil pulls out a thing of coke and my HEART JUST DROPS. The bass didn't help either.. Going from a really exciting, fun night - I went to anxious, introverted. I sat in that spot frozen until the concert was over. We left and I told him I had a horrible night when he pulled that shit out. He apologize, I forgave him. Come to the first week of January (9 months now), Phil (lives about 2 hours away from us) asked him to come chill for a night in the future (set for tonight btw) because his wife was gonna be out of town and he wanted some bud time with him. Cool. Come to today, my boyfriend forgot about it so he was gonna offer for me and him to come down and chill for a bit, but he can't stay. Phil says "no worries, I understand. People won't be showing up til 7 anyway." I was never informed that it was a party, he told me that it was just gonna be him and Phil. So if he was hiding a party from me, my mind naturally assumed coke might be involved (because it was Phil's party). Like I said, I feel like I'm overreacting and it's just anxiety. But I also feel like I'm not overreacting and that I really can't trust him.... Please, no criticism, I'm honestly just stuck in my thoughts and lost for words and just need some advice. Sorry this was so long and probably hard to follow along to. Thanks to anyone that actually helps!