BC and pregnancy?

If I'm on an oral contraceptive (and take routinely), and give bf handjob, do I have to worry about semen on his hands when fingering me?