Lube during my peak

I didn’t know that lube would interfere with sperm to reach to the egg until now. I had 2 smiley from opk, today and yesterday. Only today we used lube. Would it kill my chances with the sperm from yesterday as well? I am really worried. I told my husband not to use lube. But he was pressured since he couldn’t come inside me so he have to masturbate first.