My boyfriend can't last

I'm 21 and my boyfriend is 33. We've been dating a year and a half and have been living together about a year. When we first started dating he could last about 30 minutes, and every once in a while couldn't really keep it up. Which annoyed me a little but maybe it's because we were having too much sex. Now, when I go down on him he can only really handle it for about 30 seconds and about 5 minutes of actual sex. I don't want to say anything because I don't wanna hurt his feelings, but sometimes I'll say "wow that was fast" and he'll say "yeah it just feels too good" he's even said once that it's because he's getting old lol. Is it me? Is it him not trying to last? What is it and how do I fix it... He seems to know he finishes really fast now.