TTC - now husband uninterested

My husband and I have been trying to conceive for 3 months now. He has been wanting to try for over a year and I have been holding out but now that we are trying, he doesn't want to have sex nearly as much. Today was my most fertile day (he knew) and I swear he picked a fight on purpose so we didn't have sex. He says it wasn't on purpose and his sex drive hasn't gone down since we have been TTC. I know he isn't cheating and still wants a kid, so what is going on? Anyone else experienced anything similar? Is he just scared because a pregnancy is now possible or is it just an odd coincidence? (FYI, he is very horny my period week and week after.)