Boyfriend not being hard 24/7 ?

Lately, I've been experiencing a situation.. my boyfriend usually initiates intimate moments, but after a few minutes, he loses his erection and suggests we continue later. Later I've noticed he goes to the bathroom and comes back with his phone, which makes me worry he might not be attracted to me anymore. It's uncomfortable since he sometimes stops midway and starts stimulating himself, and again says maybe we should try another time so I can't help but wonder if it's something I've caused.