Boyfriend going soft?

My boyfriend and I just finished having sex. It’s currently 10 am and we started about 9:30. I woke him up by playing with him until he got hard and then he woke up and we had sex. He usually doesn’t wake up until about noon-1pm so this was quite early for him. So point being; he went soft multiple times and could only get himself hard again by using his hand. And then he made himself cum using his hand instead of...me? I’ve been told he could be going soft because he’s tired but it’s happened before. Where he just randomly goes soft during and has to get himself hard again before we can continue. Is this normal with anyone else?