Should I just leave?

I have the best husband one can have. He's supportive. He's helpful. He's sweet but practical. And most of all he loves my son. 
But he wasnt always like that. He was a drug dealer and was addicted to drugs himself. He would get high ( me, oblivious of what he was doing) and would convince me to have sex due to the side effect of the drug. Now that i know about it i feel used. I feel defiled. I didnt know then. I thought it was affection. He was pretty good at hiding it for a year actually. Maybe because i just trusted him a lot.  
When i found out about the other side of him. I gave him a chance offered him help and support. Offered to go with him to rehab. He said , just me giving him a chance is enough. But he still broke it. Gave him a chance again, ruined it, then again , then ruined it. My best friends think im stupid. Says i should just leave him. I stayed. It carried on for about 5 months. One time he was drunk. We argued and he beat me up. The neighbours knew and called the police. He got arrested but was on bail for a week. 
Few days later i found out i was pregnant.
He said if i keep yhe baby he will change. I cant abort him. Im against it. So i decided to keep him.
 9 months later. I had a baby boy. 
My husband suddenly changed. 
I mean seriously ? Who is he? I dont believe this. He used to beat me up. He used to get drunk every single night, he used to get high on drugs, he used to lie to me about everything.
I dont believe this. Im going crazy. The thoughts in my head, " he's going to betray me for sure" like before". Every little thing feminds me of how he used to treat me before. Because of these experience, i cant takw care of my son properly. I think i am damaged. Ill never hurt him. I want the best for my baby. But i get frustrated at him since he cries alot. 
My husband says he'll take my son if i don't change as i am dangerous around my son. 
Maybe he's right. I probably am a horrible mother. He told me that i can leave if i cant move on. 
Maybe i should..