RELATIONSHIP ADVICE

Basically, I've been seeing this guy for  6-7 weeks and he's fallen really hard for me. I feel bad, I'm not as attached. Anyway, he's never had a relationship before (he's 22..) and he has trouble getting it up. At first it was fine, we had sex often, but as time has got on, not so much (last 2 weeks). He thinks it's stress as he's 'suppressing emotion' (basically nervous over how unaffectionate I am). This is making me feel bad, I'm genuinely an unaffectionate person and hate PDA, hair stroking etc. 

ANYWAY, I've been getting annoyed over him not being able to have sex (yes, I know, so shallow) and I want your fellow advice pleaaaaaase!! 

Thank you!!

B x