Anxiety/Panic attacks surrounding bars etc

A short intro, I'm 18 my boyfriend is 20 (legal drinking age here, 19). Just like many people, I've been in bars already (blasphemous I know). The few times I've been in or around a bar I've had a bad experience.. Cat calls, up to assault/waking up in a different city with no idea of what happened. I'm not irresponsible either, I don't put drinking high up there.. That's my personal thing. But anyways, my boyfriend.. Wonderful respectful man just.. Whenever he texts me to let me know he's going to a bar with friends or something I just feel anxious instantly, hyperventilating, feel like I'm going to cry. It's not that I don't trust him, because I do. And I am 100% secure in our relationship. But it makes me feel awful.. I don't know if maybe because of what I've gone through, bars etc heighten my anxiety or what. But I genuinely feel like I can't calm down, and the feelings.. It's a mixture of anger and sadness. I guess I'm just hoping someone understands...