Toddler poop

My toddler is 3 1/2. We have been potty training for a year. He pees in the potty pretty well but has never liked pooping in the potty. If I don’t make him poop in the morning he will hold it for days until he can’t anymore. His poop ends up being green and diarrhea-like if he holds it in for a day or two. Anyone else going through this? I’m tempted On going to the doctor to see what I can do. He’s almost 4.