Could I have made him infertile??

My S/O and I have been trying for a baby for around three years now but havnt conceived once. He discovered he’s really turned on by ball busting and tbh I enjoy it too (don’t judge!) most of the time it’s light slapping or squeezing but sometimes it’s much rougher. He’s never had any damage because of it and has never had to see a doctor or A&E but I’m now worried that years of doing this may be the reason we have struggled to conceive. We are going to the doctor next week for fertility referrals so I guess we will find out soon enough if his swimmers are the issue. Just wanted some input from you all as I’m pretty sure we won’t be mentioning it to the doctor 😂