New puppy starts pooping in the house and won't finish outside.

I don't use any kind of negative reinforcement with him. The two times I have caught him pooping in the house, I just pick him up and bring him outside. I don't rub his nose in it, don't make a loud noise to startle him or anything like that. Just pick him up and bring him outside. The issue is that no matter how long I keep him outside after, he will not finish pooping.

He is doing great with peeing outside and knows to go as soon as I bring him out. His poop is normal and when he is going, he doesn't seem to have any issues getting it out.

He is in the process of being crate trained and doing well with that. If I can't have eyes 100% on him, he is in the crate. Which isn't often, mainly at night and when I am cooking breakfast and dinner.

How exactly can I get him to finish pooping outside once he tried to/started in the house if he just won't go no matter how long we are out?

Edited to add: I know it takes 6 months to a year to fully house train a dog. I've had experience with that. And I'm not worried about the accidents themselves because I know they happen. I'm not frustrated with the puppy. I am worried about his health with him holding it so long after he is showing signs of starting to go in the house or after he starts going in the house.