? About anal and TTC

Whenever we do anal I have my hubby put a condom on then right before he cums I have him take the condom off and cum inside my vagina since we are TTC.

Could the lube from the condom affect TTC?

Thanks.