Flu shot for baby

Hi all, before anyone jumps down my throat...I'm NOT anti vax. My 6 month old son is up to date on all shots except hep b (only one round behind because we waited until he was 2months to give him this instead of day one) 
His doctor mentioned that flu season is coming up and he said bub will have to have one plus a booster. I haven't had the flu shot for years, including when I was pregnant, and I haven't had any issues. Before I stopped taking the flu shot, however, I would be sick multiple times that fall/winter. I totally understand that babies are more at risk and have less developed immune systems and all that jazz, but I'm just really struggling with the idea of giving him the flu shot. The fact that I always got sick immediately after the shot and later on as well has me concerned to do the same thing to my baby. 
So I'm just looking to hear other ppls experiences with this. Did your baby get the flu shot and got sick, or was fine? Or didn't get it and got sick or not?