Why can't he stay hard

Hi - I always manage to get my bf hard and during foreplay we'll take it in turns to do things to each other. But sometimes by the time he is done doing things to me, I'll go back to touch him and he's soft again? Is it me?! I don't know...