A man's age effects fertility?

We have been trying ttc for 7 months now, both heathy, I'm 30 and he's 39. I never thought a mans' age mattered, but now I'm reading that it can significantly effect their sperm. Does anyone have facts on this? Just feel like it's another thing for me to worry about!