Partner can't last

Hey this is the first time I have ever talked about this but I am getting so frustrated now and upset I don't know what to do. I love my partner but nearly every time we have sex he comes within a couple of seconds (literally in some cases) and other times when we are having foreplay without me touching him down there he comes. I feel bad cause I know this gets to him but I am left unsatisfied and he won't play with me after to help me get off. I can get off with him playing with me then we do the deed and because I got off once when he played with me he says I should be fine. Am I doing something wrong or could there be another problem here I don't want to be selfish but I just don't see the point of doing anything like penetrative if we can't get off together and leaves us both frustrated me physically and him emotionally cause he can't last.