Mismatched Sex Schedule

My boyfriend and I are in a long distance relationship and we're only able to see each other for a few days every month. Anyway, we both enjoy sex, but we're always "in the mood" at completely different times. He's a morning person and enjoys having sex right when he wakes up or during the daytime. I'm not a morning person at all, so I'm never in the mood for sex that early in the day. However, I get really frisky in the evenings, but he's usually burned out by then and ready to chill. We've both made compromises and had sex according to the other's schedule, but it always feels like one of us is exhausted or not at their prime. Has anyone else had difficulty coordinating time for sex that works for both partners?