Is it me or my hubby whose infertile

Me and my husband have had exactly one pregnancy in our entire relationship that's 16 years. We got married in 2011 and been trying since we miscarried with our first and only pregnancy. Which was in 2010. Since then I've done multiple tests for everything that could be wrong with me. Blood work and exams and endless tests that always come back negative. I am over 200 lbs yes but I quit smoking and drinking and doing anything that could hurt my chances. After all these years I have begun to wonder if it's my husband whose got a low sperm count or maybe he's infertile. I'm so anxious now! If it's been him this whose time, I feel like I chose the wrong man to have a family with. I'm just so tired of that damn negative line!