Panic attack during accidental anal

When I was 14 I slept with a grown man who was in the army and had been engaged twice. I knew him because he used to be my neighbor, he was 5 almost 6 years older than me, and he forced me into anal and I couldn't scream for help or for him to get off because I was on drugs and wasn't clear minded but it scarred me and I got over it. He's now married and is raising his wife's daughter. Well over my boyfriends leave during Christmas we tried anal because he was very eager. And I was willing to try because I love him and I was comfortable. We've been together for 2 years now. Well he barley penetrated and I went into a full blown panic attack for 10 minutes. We agreed to never do it again until I suggested it. Well last night we were having amazing car sex (so amazing) but at a point when I was laying down he thrusted  out and when he thrusted back in wrong hole! And I had another full blown panic attack. My partner has and will always be there for me, but I feel very guilty and like I should apologize for my actions when I was younger. He doesn't make me feel like I need to apologize but none the less I still feel guilty.