Eating habits

Who else is struggling with feeding. My son has been eating on and off. Today he likes food and next he doesn’t. I tend to give him milk when he refuse to eat just to have something down. The pediatrician wants me to stop giving him milk. But do I just leave him hungry till he eats or? I am really struggling here. Guide me please mommas