My boyfriend isn’t ready to get married

So my longtime boyfriend and I have been planning and saving up money to buy a house. We both are mid twenties and in stable careers, he moved back in with his mother and his aunt in order to save money, and we both are doing pretty good and planned on getting married sometime before that. We even picked out rings and made a potential ring list, and talking about when we would anticipate getting ready to have children, it’s been a year since we picked out ring lists though l.We had a fight today and kind of out of the blue he said he just wanted to leave me and move away by himself. This caught me super off guard and although we where in a fight I didn’t think it should have been escalated to that, I got really sad and started crying and basically just walked away because I needed a minute after he said that. We came back and talked about it and he apologized profusely and said he didn’t mean that and that I was so angry he just said that so I would stop talking. It honestly didn’t make sense for him to say that things considered plus besides our occasional fight we have a great relationship, so because of that I believe him that he just said something stupid that he should not have. I still feel weird and insecure about it though and after things cooled down I did ask him if he’s sure he still wants this. He told me that he’s hesitant to propose to me because he thinks I am insecure in the relationship because I’m not 100% sure that he will be there for me no matter what. I don’t feel this is fair for him to say after what he said when he was mad. Am I just insecure and everybody says stupid stuff from time to time or am I justified? I just feel like I’m saving money and making plans to be able to buy a house with him and if he doesn’t want to get engaged at some point then I should know sooner than later so we don’t waste our time.