Worried. ☹

We have just started ttc! We already have our 2 year old son and we thought it was time to try for a little brother or sister for him. Well today was one of my fertile days and we used lube just because he's always liked the feeling of using it. It was VERY little. Like 2 little drops but I had no idea lube isn't ok when ttc unless it's like preseed. Do you guys think it could really affect it?? It didn't have spermicide or anything it was the Jo h2o, something like that. I really hope it doesn't mess anything up!