Losing erection during sex

I've never had a man lose an erection during sex so I'm kind of confused. My s.o. & I were having sex (first time in 2 months)& we barely get started & he's on top, I look up & he's looking down at me & then he stops & goes down on me while trying to get his erection back. He tells me he's sorry & that he's just missing his kids who went back w their mother for the school year. Now I believe that he misses them but I still wonder if it's something else because he was looking at me when he lost the erection.