Husbands just not getting it

We are on our 3rd cycle. We keep missing the peak days because hubby always says he's too tired. We don't have a very active sex life anyway so we pretty much only have sex for my fertile 5 days. We had sex last night which was the start of my fertile window. Today I got an almost positive opk which is like 2 days early so I said to him we should have sex tonight and he's just refusing. He's saying I am trying too hard which I understand what he is saying but if I didn't try we would never get pregnant because we never have sex! Last cycle we only had sex 3 times and the cycle before only once. I just feel like he isn't committed or doesn't understand how important it is. He wants a baby but all he does is complain he has to have sex. I've never been in a relationship with a guy that doesn't want sex. I understand he might have a low sex drive but it's making me so disappointed. I feel like I'm stressing myself out every month doing opks and monitoring my temps and it's all pointless because we're not having sex at the most important times.