Should I wake baby up at night if I want him to take smaller bottles more often?

My 4mo will sleep anywhere from 6 to 11 hours at night. On the nights he goes longer he’ll demand huge bottles during the day, anywhere from 5-7 oz and then be hungry again 2 or 3 hours later and eat 5 oz again. He never eats more than the max recommended limit for his age, not even close. But people keep telling me I’m going to stretch his stomach out that way.. I’m trying to transition him back to 4 oz more often but he’s just not having it. Should I wake him to eat? Or will that cause him not to trust his own hunger and sleep cues?