To go to work or not to go to work...

My husband and I are still trying to have children but in the mean time unforseen circumstances have brought us adoption of our wonderful son. He is 16 months. But my dilemma is this... Financially we are "ok". I am tempted to go back to work to help out and start saving again but at the same time I don't want to leave the baby.... advice, personal experiences, anything would be helpful. TIA