Please help! I want my husband back!

My husband and I have been married for almost two years. We have a 4 month old daughter. We have been arguing non stop about the same ole stuff for about a year now. He had an emotional affair on me and I'm having the hardest time letting it go. I bring it up all the time and I have major anxiety that he will hurt me again. Not only that, but I'm going through depression, and he just doesn't understand it at all. He says that I nag him and treat him like a child. Also says that I don't appreciate him. About a month ago, our argument was the most heated it's ever gotten and I left. We both said some hurtful things, but I just said things out of anger. I made it clear to him that I didn't want a divorce, but just wanted us both to have time to cool down. I came back home Sunday night, and ever since, he's been extremely mean to me, I can't even talk to him without him being so mean. For the sake of peace and to try to get past this issue and save our marriage, I told him it was all my fault and that I wanted to work things out, and he says he doesn't know if he wants to stay married to me. What can I do to win him back and make him open to fixing these issues? I love my husband and I am so afraid of losing him. We can't just get a divorce just because things got hard. Please help!