Can I get in trouble?

So I had a baby boy 3 months ago and I am taking a trip driving to Arizona at the end of the month just for two days however my ex is saying I can’t take him out of state without permission? Neither of us have established custody we have just been co parenting since he was born. My son has lived with me since he was born also. We are not married.