Need help

What should I do?

My ex bf is moving out of the school district of our 4 year old son to live with a married women he’s been seeing for a few months. We haven’t ever been to court to talk about custody or any of that. We work the opposite days so it works out but I feel this is totally wrong. He’s not doing what’s best for the child, but what’s best for him. Any advice?