Help please

Im in my first serious relationship ever and at first it was the best thing ever. I could trust him and I knew he actually loved me enough not to do anything. Over time though he lied about little stuff and it slowly broke down my trust. He claims that im controlling and im the reason why he doesnt hang out with his friends and i don’t appreciate it. Like for example he didnt go camping because i told him no. The only reason I told him no was because people get high and drunk and if he does something when hes not in his right state of mind, it could ruin what we had. And they get SUPER drunk and smoke atleast 5 blunts. Anyways prom came around and the seniors usually go down the shore. We are juniors but some juniors go to party too. Apparently im the reason he didn’t go when he didnt even ask or mention it to me? Another example is he says he doesnt have any females friends cause of me but we’ve only fought about the FRIENDLY girl friends he has. Either way tho i never told him “cut her off” and NEVER told him to block anyone either. Anyway my problem is, how can i stop being so controlling? We are currently on a break rn cause i feel like im toxic for him if i am as “controlling” as he claims.