my husband and I want to have a baby...but every time I are in my fertile week he is always tired

I am so annoyed! We have been "trying" to get pregnant since may....but every time I am in my fertile week, he is always tired from working. 
At his point I think we will never get pregnant. I have told him early in the day that I want to have sex tonight and then by the time I get home from work he says "I'm sorry I'm too tired" but last week when we had sex we climaxed at the same time and he said "you know what that will probably be pregnant"......ummmm no! 
I told him that we need to have sex during my fertile week and he agrees but every single month on my most fertile day we have never had sex!!!! 
I do not know what to do, or what to say to him.