Anyone not taking prenatals while EBF?

I realized I haven’t really been taking my prenatals while breast feeding. I stopped taking my prenatals when I was 14 weeks pregnant because they made me really sick even if I took them at different times of day. My dr said it was fine if I got my vitamins through food. But Idk if it’s doing damage to baby boy taking them while breastfeeding. Baby is gaining weight on track, and is on track or a little ahead so I wanna say it’s ok. I give baby vitamin d drops as well