Married, not in the mood, husband is not understanding...

My husband and I have been together for 12 years, married for 10. I was diagnosed with PCOS and we've been trying for the better part of our marriage. For the last couple of years, my sex drive has totally tanked. It's not because I don't love him, or that I am cheating or anything of the like. My husband feels just because he's ready to go at any moment of the day, I should be too, and that's just not the case. He is constantly pissed at me because I am just simply not in the mood, and he just makes it worse by making me feel like a peice of shit. I don't like being grabbed on like a peice of meat or constantly reminded that I m never in the mood, I already know this,I live it everyday and don't need reminded. I am trying things to change this, one being thing on a healthier lifestyle, exercising and trying to loose weight,as we all know a factor of beating pcos and getting pregnant, also help relieve stress from my job. But I must also mention that we don't have the most ideal living situation at the moment. We live with my parents because we are completely remodeling our house , and it's not exactly the most private situation. We had to install a lock on our door because no one understand knocking and waiting for an answer before you enter someone's room. Ugh, I don't know. I don't think this is a deal breaker, but he sure acts like it is. I m tired of him being mad at me ( as we speak he giving me the silent treatment and playing Xbox ). Don't know what to do.