Advice!

Okay, so me and my boyfriend have been together for 6 months. Well last weekend we went out and we’re drinking. We were at the bar for a while and then we headed to my sisters house. Her boyfriend does drugs every now and then just for fun and I have no problem with that because I’ve done that too but now I can’t do drugs like that because it fucks me up. Well my boyfriend knows that and he doesn’t tolerate drugs and I choose not to do them for him. Well when we were over at my sisters I went to the restroom and my sister was in there so I was in the bedroom. (They live in a very small apartment) well her boyfriend comes in and asks if I want to do some stuff that he had and I said no that I don’t do that shit. My boyfriend overheard what my sisters boyfriend said and got so mad at me because he thought I had done them. I told him that I didn’t and I’ve told him that I don’t like to do that stuff and I haven’t done it since we’ve been dating. He’s accused me of doing drugs before and I just feel like he doesn’t trust me.

I just don’t know how to gain his trust because I haven’t done anything to break it. Idk what to do and now I’ve been thinking about all of this stuff and I’ve been acting weird towards him. We’ve talked about it but I still feel weird idk. What do I do???