What to do? So confused

So my fiancé and I have been together of and on for almost 4 years. We have two kids together and we love them like crazy. I have been waiting for him to pop the question for a bit but I knew when he felt ready he would. Well back in March he asked. I said yes (obviously) but maybe a couple months ago I started to get serious about at least picking a date or a year to get married. He then tells me he doesn’t want to get married yet because he is afraid that one day this won’t be what he wants anymore and we would have to get divorced. I asked about when he proposed and he said he was happy and was scared cause we were expecting another baby and kinda did it on a whim. Then a few nights ago I brought it back up...it makes me wanna leave if he is so unsure of everything. He had been drinking when I brought it up and all he said was “I can’t give you the wedding you deserve and it’s embarrassing” he knows I don’t want a big expensive wedding. I feel like if he has changed his mind from wanting it to now isn’t sure then it’s hard to want to stick around. I love him but I don’t get the attention I deserve and our kids don’t either. He is always on his phone. I do basically everything. Maybe he figures why buy the cow when you get the milk free?