21 weeks & Having issues

My husband and I haven't had much sex since I've gotten pregnant. Lately it's almost non existent and I mention it to him and he just says he's tired by the time I get off work. I work from home but get off at midnight and he makes it home around 7pm. I still try to make his dinner and sometimes he will cook and we take care of the two dogs. But he use to want sex even if he was halfway asleep in the middle of the night! it's not my first time working a night shift so I don't think that's truly it. Then he says he sometimes worries about the baby but I honestly think that's an excuse because if he is really in the mood the baby is the last thing he is worried about. And then there are times when he is in the mood, which isn't lately honestly, and I'm not or I'm in the mood and he just isn't, which seems like it's more often then anything else. Sometimes we will be having sex and he just doesn't stay hard or won't finish. Like tonight, he came home early so I took off the rest of the night so we can have us time but still nothing. I mention it to him all the time to keep our communication open but there is always an excuse. He says things will go back to normal after baby but I'm not so confident. Does anyone have any advice? I'm so confused on what to do.