When do you move on?!

Katie
Our story is much too long to be summarized without losing others interest. Here are the bigger topics that summarize where we are in our fertility journey:
-Male factors. Husband had cancer as a baby and had a testicle removed. He is healthy as an adult (34), but his testing showed last year all not great.. low motility, not great morphology, low count. 
- I have NEVER had regular periods and have seen 3 specialists. I have undergone ALL testing available and have spent tens of thousands of dollars. My eggs are "fine", but never grow beyond about 20. Reserve is good. Hormones are up and down. Throughout medicated "forced" cycles, my hormones are fine- unmedicated they are up and down. No significant issues to mention. Tubes are not blocked, I did the blockage dye test last year. 
- We did 3 <a href="https://glowing.com/glow-fertility-program">IUI</a> cycles back to back to back last year. After the $ and the heartbreak, we took a break. The break lasted longer than we'd like, but life tends to happen. 
- I have (in the past during our "trying") been on Femara (highest dosage possible after several months of increase), Estrogen supposetories for my lining because it tends to be a bit too thin, and progesterone to even get the cycle started... I also did trigger shots all months of <a href="https://glowing.com/glow-fertility-program">IUI</a> (3) and triggers two other months with medication but I <a href="https://glowing.com/glow-fertility-program">IUI</a> prior to the IUIs. 
- When I say all of our siblings are pregnant right now, I mean literally ALL. Both of my sisters are pregnant (well, one just delivered my sweet nephew 2 days ago), and his brothers wife is pregnant. 
We have been getting more serious about starting <a href="https://glowing.com/glow-fertility-program">IVF</a>. I'm passed heart broken. Now, I'm super happy for everyone around me who is expecting- but I don't know how much longer I can sit back and not get our "piece" of happiness. We want a little one to kiss and snuggle and love. We have waited so long. 
My real question is: since we have been not "trying" (I have zero regular cycle in my own. Literally maybe a period every 3-5 months and it's a 2 day spot) for about a year now after all the failed treatment, so we get back up on the horse and back on meds first before going full throttle <a href="https://glowing.com/glow-fertility-program">IVF</a>? Is it a waste of our time?! Why waste more?! I just don't know anymore. Anything you can say is worth it, I promise. I just (how, I have NO idea) started an AWFUL cycle of horrid cramps where I had to literally benedryl myself to sleep for 12 hours yesterday to get through and felt like I was seriously IN LABOR and had clots yesterday and this morning. It's over now and only lasted hardly two days. I am just at such a cross roads, but worse, I'm confused. 
I went to the two largest and "best" fertility doctors in the Midwest for our first treatments with <a href="https://glowing.com/glow-fertility-program">IUI</a> and our years of testing and trial/error. There is one more doctors office we want to try, where multiple friends of ours achieved <a href="https://glowing.com/glow-fertility-program">IVF</a>. (Midwest Reproductive Group).