Question!

Is there such a thing as being fertile with one person but not another? Could his sperm and your egg just not work together for some reason?