Do I HAVE to brush babies teeth after EVERYTIME he has milk?

My son just turned a year old. I am now giving him whole milk. I give it to him in a sippy cup at breakfast. Then we go upstairs and brush our teeth. Then give it to him in a bottle before nap time and bed time. I brush his teeth before bed time too, right after the bath and before bottoms.

Hes used to drinking milk with the lights off and then going to bed right after. I figure if I change the routine he may not cope well and it’ll be a challenge to get him to sleep.

So my question is do I have to brush his teeth right after he has milk (before nap and bed time) or can I just give him water right after? I heard you shouldn’t let milk sit on teeth because it can rot them. So I’m just wondering if I should change the routine to milk, brush teeth, then book and bed. Or just keep it how it is and just give water.