Is my husband having an affair?

First of all we have been TTC for 5 months because he was finally ready. Last month he totally avoided me during my fertil window. Since then he's not had Alex with me at all. He says it's no big deal and I need to stop reading too much into it. The times before this when we did have sex I had to initiate and always have to. He shows no interest in me sexually or otherwise. I would think it's because he had changed his mind about having a baby, but he's always talking about how bad he wants one. I'm so heart broken. I tell him I'm lonely and need him to show me he loves me. He responds by asking me why I can't just be content with how things are. He's recently started a new job abd there is a Melissa he talks a lot about. When I mention that I think he's having an affair he does not deny it and, he's not get upset that I would think that. I have asked him to do little things to show he's thinking of me, but again, he says he will but he does not do any of it. He does not do anything to help around the house because he works full time. He also can't have sex because he's tired from working. I also work full time, clean the house, run kids all over the city, run my husband to work and pick him up. I do all of this getting up at 4:30am and not getting to bed until midnight or later. He blames me for not getting enough time together because I don't take a long enough nap after work so I can stay up with him at night. What do I do? It sure seems like I do a lot but I'm not doing any of it right. ?