Infertility ruining marriage?

Does anyone feel like infertility can possibly ruin your marriage??? Backstory my husband has 3 kids already and had no trouble conceiving them. We’ve had 3 failed iuis 2 years of ttc with no luck and I’m just at wits end. It’s hard to deal with it when he already has kids. He says us having trouble bothers him just as much as it does me but I don’t see it that way cuz if we never had kids he would still have kids. I just don’t know what to do anymore, some days I wanna just take a break is anyone in the same boat that can relate?