need some advice

Me and my husband have been dating for over a year now got married 5 months into the relationship we felt we truly loved each other and we couldn't wait. I have two children from a previous relationship and he has a child as well. Things started getting tuff wen we decided to move in together at my parents since we couldn't afford living on our own. Things got really tough living with family and have bad depression I tried very hard to keep my self motivated. We started arguing more about the kids and family who just kept getting involved. He recently told me he does not want to continue this after we spoke and decided to move out to work on us and the kids but the next day he busted out with I don't want to do this anymore and wouldn't be fair if he was not honest and made us move. I'm torn apart he's wanted to make a peaceful relationship to pay our debt together and see what will happen with our baby that is on the way. But I just been so furious and hurt I haven't been to cooperate with him. I just feel he has given up so fast after he promised me he was ready for this. Idk what to do I don't want this to end I feel like driving to his moms and wait for him and show that I'm fighting for us. Is that wrong?