Goes soft

Almost every time we bd he goes soft. I think it's pressure of it all but he's the one who brought up having a baby so I'm not sure why that would bother him. It's frustrating because we have such a small window. I'm not sure what to do. It doesn't happen during sex when it's not during off peak time.