My husband doesn't want to have sex..

I am 24 and my husband is 34. We got married in August last year. We have been together for a little over three years now. During the first two years of our relationship we had sex normally and a few times a week. For the past few months we have scarcely had sex. I try multiple times a day to have sex with him but he keeps turning me down. I understand that he may not always be in the mood but it is happening too frequently for it to be only that. I keep getting excuses that he's tired, it's too late or that he needs to go do some other task. I know he isn't cheating so that's not a reason. And I haven't changed my physical appearance at all and I believe my attitude and personality to be the same. We are TTC which he really wants (as well as me) but it still doesn't help with the frequency of our sex. I have asked him about it and he doesn't have a good reason. I don't know what to do. I think it's bad for our relationship and my self esteem. Any advise would be helpful!