My struggle......

My husband and I have only been TTC since May which is proving to have its problems.  Here's my story.... DH and I have been married almost 3 yrs, if I would have agreed we would have started trying before we got married but since I'm super traditional and come from a traditional family that wasn't happening.  
​Any how, we use to have knock out fights about starting our family until this May when I decided to take out my IUD.  I keep him informed about my fertile days and my when AF is suppose to show because he initially said he wanted to know and that he wanted to help me through this process and do his part.  He even added the app on his own!!! Well, now we get to those critical days where it's imperative to BD on those high fertile days and he won't finish the deed!  
​I get so frustrated because I often feel like is it me and he just can orgasm with me or what?! We talked and he said he felt pressured, so I conceded and said I wouldn't tell him about fertile days or anything that I would be spontaneous when it came to him (which I usually am, morning sex to wake him up, blow jobs as soon as I walk in the door, I sometimes even leave work early to give him a BJ at his job if I know he will be home late, the works!!!) we had sex almost everyday of my fertile week and he only ejaculated twice!  
​Even the times that we know I'm out of my fertile week and we still have sex he won't ejaculate, EVER!  My husband can be very affectionate but over the past two days he's been very stand off-ish towards me and not really talking to me at all.  All I get is hey, bye, or what time are you coming home.  I ask him what's going on and if there is anything on his mind but he just says no when there is clearly a disconnect.  
​ Initially I was scared to start a family, we weren't in the best of financial situations (I lost my job, he lost his) but now we are back on track and I'm all for it now he's the one holding back.  Please help me shed some light on this because I'm all the way in the dark without even a match to give me small glimmer of light.