Boyfriend can't keep erection?

So last night my bf and I were going to have sex. Lots of foreplay and he would get hard for a couple seconds then go soft. I kept giving him head, kissing, touching, he went down on me, and he couldn't stay hard long enough to go inside me. This morning the same thing happened. And this morning too I was giving him a bj, he got hard, I continued to give him a bj, and he went soft. He was very upset and frustrated. I felt so bad for him. This is not the first time this has happened, usually happens a few times a month. He's 28, healthy, works out everyday, and not under a ton of stress... we both don't know what's wrong. Help!

- He does not drink, has never done drugs, and is against prescription pills. He is all into eating healthy and all natural which is the way he grew up. He even hates it that I take bc pills because he feels like it's bad for my body. When I'm sick, he is always trying to find herbal remedies because he doesn't like all the cough syrup