weird trigger

Okay so I have a really weird trigger. It's weed. To give some back story I've never been interested in smoking, and I never had much of an opinion of it. I don't necessarily care if other people do it or anything as long as it's not around me or at my house. You do you. But I dated a guy who gave up smoking to date me (which I never asked him to do) and almost 2 years into the relationship I found out he ditched my nye party to get high with his friends. And that's pretty much where the trigger started. He was an asshole anyway but I guess i associate the weed with the anxiety from that mess. I don't wanna go that much into detail. 
And like, it's not a super terrible trigger. I have friends who smoke and they also respect that it's just not my thing. It's really an internal thing. If weed is brought up its like a pang of panic that quickly goes away. But I just feel so alone. I feel like it's a ridiculous trigger but it is a trigger and I really can't help it. I'm not being judgmental I just really don't like to talk about stuff like that bc it makes me feel anxious. And I don't believe weed is a "bad drug" I think it's great that it does help people. 
I'm not necessarily asking for advice on this. And I'd really appreciate it if people didn't recommend me trying it that really negates the point of this post. I would like to find someone who can relate if possible. I just feel very alone. Thank you for reading this also. Means a lot.