Long term traveling with kids

My husband and I have talked a lot about our dream to see the world once we have kids. We had to wait many years to have kids for health reasons, so I guess our newfound freedom has given us a new perspective on life, ie. we don't need to conform to the "norm" of 9-5 jobs and sending our kids to school. Instead, we are looking for ways to travel for a few months a year (or longer), on a budget, and would homeschool our kids and give them the valuable lessons of exposing them to different cultures. I'm just wondering how early is "too early" to start - we have some friends who waited until their firstborn was 6 months old to them do an extended road trip with a trailer, while others have waited longer than this. I'd love your advice, from anyone who has done extended traveling with kids, how you made it work etc!