Need Advice bad!!! Sorry long post!

Chelsea • TTC baby #1 a little over 5 years. I had an early miscarriage in March 2015. Hoping we get our BFP soon!!!

I was diagnosed 2 years ago with stage 4 endometriosis. I had been ttc for 3 years prior to my diagnosis. I had no symptoms of Endo other then not conceiving. My doctor actually didn't think I had it because my periods were normal, I had no painful cramps or intercourse, and I had conceived about a year after trying, but had a miscarriage. So anyway I had an HSG before my surgery to make sure my tubes and uterus were clear and they were. So during my surgery is when my doctor found I was full of endometriosis and actually had a large endmetrioma on my right ovary and 90% of it had to be removed. So my doc said best chances to convince was through <a href="https://glowing.com/glow-fertility-program">IVF</a> well we weren't ready because of the cost so we decided to just keep trying on our own well 2 years later and still no babies. I have had many breakdown's through out this 5 year journey of ttc and my husband has been my rock and would tell me "it's ok babe it will happen eventually" Well tonight was a whole different story, he pretty much said he was done with me because I couldn't give him a baby and that it's all my fault that I can't get pregnant. I was shocked, hurt, and pretty much heart broken after hearing those words. All these years those were my break downs to him saying it's all my fault that I can't do the one thing I was made for and he would always tell me it was going to be ok. so after this of course I lost it, cried, and told him how aweful it was for him to say that to me. He apologized later and said he hurts too and that's why he said it. I get it men feel pain about it too, but never would I have thought he would say those things to me. Should I let this go or should I talk to him more about it? Honestly I have just put my emotions aside for awhile now by not talking about it and he said that's why he thinks I don't care because I don't obsess about it like I used to. it's just easier this way for me it makes me feel less depressed so I don't think about or talk about it. Have any of you gone through this or anything similar? Thank you for any advice!