Caffeine and breastfeeding

I cut out everything with caffeine during my pregnancy. Now that baby boy is finally here,I was looking forward to drinking my tea and coffee again however,I’m worried about the caffeine passing through to the breast milk. How should I go about timing the drinking and breastfeeding to avoid this?