Is it true that you lose the baby weight quickly?

Someone told me you lose weight quickly because you're rushing around taking care of baby. I'll be breastfeeding and a stay at home mom...this doesn't sound right to me. I'm afraid it'll take me a year to lose as much as I want. How long did it take you from experience?