giving baby water at night?

So my son just had his 4 month check up at pediatrician and he's growing beautifully. Weighting in at 18lbs 15oz and is 27in. I mentioned that he still wakes up a few times at night and she suggested we take away his 4am bottle/nursing session so he'll start sleeping through the night. She said it's not detrimental to him at all because he's growing very well. That he's getting up out of routine and not because he's starving. So it would benefit us and not hurt him. He'd just be extra hungry in the morning of he's get a bigger bottle ( usually gets 4-6oz) anyways, I was given the suggestion of giving him water in a bottle rather than taking the bottle completely away. That way he'd slowly choose not to get up because he's not getting milk or nursed. I was holding off giving him water til 6months because there's no nutritional value but since he wouldn't be getting milk I'm thinking it would be fine for now... At least to try for now. I'm hoping it will be easier when he's 6months and I'm able to introduce solids. Until then this is the only suggestions i would consider.