Conflicted

My husband is a good to me about 90% of the time. The other 10% we are fighting and yelling at each other over stupid stuff. But he is a horrible father about 90% of the time. The other 10% he is amazing. I'm considering leaving him because of the way he treats our children. He's constantly yelling and threatening spanking them. The other night my 2 year old son was throwing a temper tantrum because he was over tired (we had gotten home late from the grandparents house) and my husband slapped his face because he wouldn't stop crying when he was yelling at him to stop. My son loves his dad but now will hide behind me when Dad gets mad. I constantly feel like I'm having to protect my kids. I've been trying to talk to my husband about this but it doesn't seem to really sink in. He hates his dad for some similar reasons. I'm just not sure what to do anymore. I also worry about divorcing him then having to send my kids to be alone with him without my protection, I feel like without me around to stop it he will just get worse. Any advice?

EDIT: After a few comments I want to make it clear that I don't stand for his behavior torwards our kids. With the incident the other night I was in another room and could tell by the way my sons crying changed. I ran in there and asked him if he hit our son (son was holding his face) and he told me yes. I then yelled at him to get away from our child and I got him calmed down and put him to bed. After the kids were in bed I found my husband and yelled at him and we argued about it for some time. I did tell him at that time that if he was going to keep treating our kids this way I can't stay with him.