Sex troubles

So my boyfriend loves smoking weed. He's been doing it for a while. But lately it's been really hard having sex with him because he starts off hard and then gets soft. I sometimes feel like it has to do with me even though I know it probably isn't. I really think it's because of the weed but every time I ask him about it he says that he doesn't want to quit. I really love him a lot but the sex isn't the same as it was before. Anyone have any ideas on what to do :(((