Husband maybe not ready after all

My husband and I have been talking about when we would start TTC pretty much since the day we got married, maybe even a little before. Our goal was to be trying and/or pregnant leading up to our 1 year anniversary, which is in October. I was on the pill until the middle of May, and now we are starting to try. I'm getting a little nervous about him though because he jokes about stuff, like "are you sure you want to get up a bunch of times a night with a kid because getting up with the dog is exhausting" Then he says he's just kidding and he's ready, but then I remind him I'm in my fertile window now so we can really get down to business, and he seems totally uninterested in sex. I should maybe try harder to initiate it myself, because maybe he just thinks I'm too tired after work, but I don't want to push the subject too hard and stress him out about TTC if he is already starting to maybe change his mind. I know I need to just talk to him about it, but I just needed to vent a little first. Am I being crazy for worrying about this? Any tips for starting the conversation?