We talked about divorce

My husband and I have been married three years and have a four month old baby. It took us a while to have the baby after having a couple miscarriages etc. Ever since my miscarriages I have been very resentful and we just really haven't gotten along like we used to. He is very closed off emotionally and I am probably overly emotional, so it's a bad combination. 
Now that the baby is here things are even worse. He barely talks to me. He pretty much doesn't look at me, and if I say anything to him all I get is a blank stare back. Any time he talks to me I now shut down and think everything he says is stupid and insensitive. I was also diagnosed with PPA and PPD when the baby was five weeks old. More than anything I needed him to be there for me and it just hasn't been that way. 
So last night I said we need to talk and I told him I'm miserable. He said he was too. I asked what we should do and he said, "part ways I guess". We talked for a long time and decided to work on things, but I'm really concerned that he jumped to divorce so quickly. 
Any advice or ideas on what to do?