Help- i dont know how to get over this

My husband and i talked about wanting children both before marriage. A few years together later he changed his mind and said he didn’t want any. I was so devastated, Ive been with him since i was 19, only man I’ve ever been with. I did end up pregnant still, he told me he didn’t love me anymore, he told me to get an abortion and said he doesn’t want children with me and i should abort and move on. I refused, we didn’t talk for a few weeks. He came back and apologized and said he was just depressed and scared. Our daughter is 3 now but i just cant get over those words. He is a really good dad, very patient and loving but in argument he mentions how he never wanted children but now we have two, i had our 2nd baby last year. He goes back and fourth, how they are a gift and thanking me for giving him beautiful children and then when he gets upset he says i was irresponsible and that he never ever wanted children. I feel like leaving will rob my children of a dad but i am so resentful now. Even on happy times i look at my husband and i don’t like him too much.