Please help

I was living in Pennsylvania with my baby's dad no family or friends over there .. he was verbally and emotionally abusive.. one time we got into a physical altercation where he put his dog to try to bite me and he took my phone and broke it in the streets... hes also a drug addict and we have a 3 month old baby and I decided to move back to ny and never go back there again.... theres no custody order or anything... i just simply came back to ny to get away from toxic environment and to provide my baby with a better future here with my family ... can i get in trouble for leaving the state with my baby without his permission?? 😔 he keeps threatening me saying hes going to have me arrested saying I kidnapped the baby