Am I doing what’s best?

So I’ve been with my boyfriend for almost 6 years and we have two little girls together. We moved in with my mom in August of 2020 so our place could get fixed up. Well here lately things have been terrible. He goes days without coming home or answering the phone or comes home close to 10/11pm and leaves at 5am. It’s like I’m a single mom but in a relationship. My youngest (10months old) doesn’t even know who he is and my oldest (3years old) cries because he won’t spend time with her. My grandmother is planning on moving about 2 hours away and I’m seriously thinking about packing us my girls and going with her so I’ll have help and be able to get a job and support my kids. Am I doing the right thing?