Problems with my boyfriend?

So when I'm masturbating, I kinda like to do it a specific way, and I'm usually on my own. And when I'm with my boyfriend he loves it when I touch myself for him, and I love it too! The only problem is, I don't really like to be touched when I'm masturbating because it makes it a lot harder for me to cum and it just makes me a bit irritable to be in the moment and hen be touched. But sometimes when I'm doing it my boyfriend will do something like touch my nipples, put his hand over mine, kiss me, etc. and it just really throws me off and makes it take a lot longer for me to actually finish. I know that he's trying to make me feel good, which is why I don't wanna say anything to him about it, but it makes me really not want to do it with him anymore. Everything else about our sex life is great, its just when I'm trying to cum that this is a problem. Does anyone have any advice for how I could approach him about this without hurting his feelings or making him feel as if he's done something wrong? Or does anyone have any advice for me to be able to deal with being touched during?