💔 should I quit my job?

My husband sucks at being a dad.. I just started a part time job. I was gone for work for 5 hours, I call home and he tells me that he's been crying for 5 hours straight. I come home and my son has a raspy voice and is very clearly upset. He is 4 months old. He doesn't deserve this. Should I quit my job? It's not getting better. They're not finding "their rhythm". I've been working for two weeks now and it's the same thing every time.