Frustrated with sex life

Me and my husband are ttc but he just isn't into having sex, I keep telling him we only have a small window that will give us a good chance. I think it's mainly because he is stressed over the whole thing with us trying for over a year. We are now seeing a fertility dr and found out he is the problem so that just added more stress. Has anyone eles had these issues?