actually really embarrassing

My s/o is a lot older than I am, like I'm 20 he's 38, every time I go down on him when he's hard his penis gets soft after like 10 minutes, he tries to play it off but I notice it all the time. I don't know what's wrong and I don't know if it would be rude for me to bring it up to him???? HELP!