When will the desire for sex return?

I'm currently 3 weeks 6 days pregnant. I've been dealing with slight nausea already, my breasts are tender, I'm utterly exhausted, and I could go on. Worst of all, I'm not in the mood for sex. I don't believe my partner understands fully as he keeps trying to have sex every day.

To add to it, and I'm probably being too cautious, this is our rainbow baby and I'm afraid that we could mess something up. I can't help but be overly cautious and I think that puts a damper on the desire as well.

If you've been in a similar situation, when will the desire to have sex return? I honestly like sex, I just am not in the mood for it.