going against doctors wishes.

So my son was 4 1/2 months when the pedi said it was time to start solids. She said at four months it was time! Umm he can't even sit straight in a high chair. We tried it once and he really is not ready yet. I'm waiting until 6 months or later depending on his readiness. Is that bad to go against her wishes?