Exercise after baby?

When is it safe to exercise after baby? I know the official rule is to wait until your cleared by your dr after your 6 week check-up, but wondering what the harm would be to start now.. I am 3 weeks pp, uncomplicated vaginal birth and my bleeding has already stopped. I feel like I’m ready but I am still waiting just in case 😩