Im not his soulmate

My husband and I are high school sweethearts. Been together since 2012. Got married early 2016 while I was pregnant with our first baby. 2018 we had our second and 2020 we had our third babe. My husband always drank, since the day I met him. He was a responsible drinker and would drink when he was at home only around his parents or someone over the legal age. A few years into our relationship I found out he was doing drugs while he drank, I stayed with the condition that he had to stop. I never dealt with anything like that so I thought everything was fine when he agreed he was going to stop and I was young and dealing with my first boyfriend. We had issues on and off about the situation and he started getting irresponsible with his drinking. When we got married he would leave to drink when we got into arguments and sometimes wouldn’t come back for a night or two. I forgave him time after time because I wanted my family to workout. I STUPIDLY brought two more kids into it. I love my kids with my life but I feel like an idiot now for not realizing I shouldn’t of even gotten pregnant the first time because of all the issues we had and now my kids are in all this mess. A few months ago I grabbed my kids stuff and them and I left without telling him anything while he was blacked out of all the drinking he had done that day. I was gone for a few weeks and he became sober and we started doing couples counseling via video bc of corona. He’s been sober since and things were good. Until a few weeks ago that he started making comments here and there about how he wished we could have fun like his friends do with their wives. Have dates and go out for drinks or just simply drink at home and watch movies. I’ve told him how I feel that we cannot do that just yet or ever because the last few times he drank he was very verbally abusive towards me, he would pick fights with me for nothing and then leave while he was drunk and would come back home and continue trying to fight even after I would tell him to stop and ignore him. This was happening EVERY.SINGLE.WEEKEND. I’ve been thinking so much about our relationship lately and I just feel like I’m not the person he wants in his life. He’s mad most of the time he’s starting to make comments here and there about the drinking he complains a lot about me not being “fun”, keep in mind im a SAHM of 3 kinds under 4. I love him even after everything but I just don’t think he’s happy with me and I don’t know how to even start the conversation with him. I’m terrified because I have a strong feeling that we are going to end up divorced and I know it’s stupid because we’ve been through so much crap and it’s easy for someone to tell me to just leave him already but it’s been so many years of working to make this work and creating a family and he’s finally sober and then just to realize it was never really meant to be. It’s so heartbreaking and I just feel so lost.