Divorce/custody

My husband and I haven't been on the best of terms the past 5 months or so. We have a 1 year old.

We agreed before I even got pregnant that I would be a stay at home mom.. it's been no issues. I do EVERYTHING with our child and the house. He doesn't have to lift a finger once he gets home from work.

However, today he brought up the word divorce. But when bringing this up, he said he would take me to court and prove me to be "unfit" because as of right now I have no job.. can he actually do that?