Ptsd and intimacy

I've started dating a guy that really like. The problem is he is very physically affectionate always trying to hold my hand and kiss me. I know for other girls this wouldn't be a problem except when I was in college I was raped. Now I have flashbacks and mini panic attacks whenever someone touches me. I think he's starting to get impatient and annoyed with my issues. What do I do?