Sex after rape.

Cierra
So last July I was raped. This year I have started my first sexually active relationship. The first time we had sex, I initiated it and it was GREAT. I felt loved and protected and he made me feel safe. 
The second time however, I wasn't exactly in the mood but I knew he was really turned on and I wouldn't exactly mind if we did have sex. Round one went fine but then he tried to go for it again. He fingered me first, which was completely okay, but then he tried putting it in and I started having an anxiety attack. I could barely breathe, it hurt, I couldn't look him in the face because I saw my assailant, stuff like that. He stopped and comforted me, but he wanted to know all that happened, but I just couldn't tell him. 
Is this something that's going to happen the rest of my life? I care for my boyfriend dearly and so far, I like sex. I don't want this to happen often.