The doctor told me we had too much sex!!!

I went in for my 30 day check up because I'm currently taking femarand when I gave the doctor my ovulation chart he said he was concerned because it looked like we had too much sex before my ovulation day. This is I ly my second cycle taking femara so I'm still trying to pin point my ovulation day so the other doctor I normally see told me to have sex every other day from CD9 until CD 20. I didn't think it was possible to have too much but apparently have too frequent sex or ejaculation causes unhealthy spermcausing less chances of fertilization.hopefully that wasn't the case...until I'll try and patiently wait for my blood results till Monday.