Husband wants to leave

So I'm 4 months pregnant and today while talking with my husband about how I feel he's been less affectionate lately he basically tells me that I drove him away etc etc and he's not happy. He's been feeling this way for a while but kept hoping it would get better. I said well then why the hell would you marry me if you weren't sure? Or buy a house with me? Or decide to have a baby? None of this makes sense and I'm devestated. Has every happy moment we've had lately been a lie? Why wouldn't he have talked to me about any of this before it got this bad? I'm pregnant with this mans baby and now he wants to leave? I'm so heartbroken. 
*edit* we've been together for 6 years, have lived together for 5 years and have been married for 1 year. We just bought our own house in 2015*