Late night sex?

So my fiance and I both work usually until 10pm. After we get home, eat, shower watch some tv it's usually around 2am that we go to bed. My question is does it matter what time we bd? We have been ttc since June 2019. Should we start doing it earlier in the morning? Or does it really not matter as long as it's during my fertile week?