When to move on to IVF?

Chrissy • TTC #2. 5 failed IUIs + 2 IVF. First born got pregnant on first IUI.
Hi, we've been trying for about a year and have done 4 <a href="https://glowing.com/glow-fertility-program">IUI</a>'s with no success. 
Thankfully with our son we got pregnant on our first <a href="https://glowing.com/glow-fertility-program">IUI</a>, he's 2.5 now. 
I know after 3 or 4, odds start declining but we've been nervous about <a href="https://glowing.com/glow-fertility-program">IVF</a>, how bad are the shots? Did anyone feel nervous about the increased risk of birth defects? 
I knew someone that got pregnant on their 4th <a href="https://glowing.com/glow-fertility-program">IUI</a> but don't know anyone that's done more than that. Has anyone done more than 4? Or for people who have done both when did you move on?