awkward sex

Okay. Last time I had sex, I was on top. This is a rare occasion, but anyways, I was riding him and he almost fell asleep.... then he couldn't keep it up, so we chilled for a second and then tried again. He again couldn't keep it up. He said that it was because of his hardcore diet... we've never had this problem before, but it made me feel like I wasn't doing something right or I was boring him.... Is this common in men who diet?