Newborn rolling to side

My newborn is 2 weeks old. He keeps rolling to his side even after I move him on his back. Should I be worried?