Time of day to have sex - does it make any difference for TTC?

I am just wondering if it happens to matter what time of day to have sex when you're ovulating? Or if it matters, as long as it's on your fertile days?