Infertility treatment

Went to see my normal ob Gyn on Monday after my husband and I agreed it was time to take action and doing whatever it takes after 4 years of trying and no baby. We had an ectopic pregnancy in 2014, and my ob said that she wanted me to go see a specialist based off of the surgery report from 2014. I came home and told my husband what I was told. He is now all upset about what it’s is going to take especially if it starts cutting into major finances. We are 26 and he thinks we shouldn’t be doing any kind of surgery until we are in our late 30’s if that is what it takes. Someone please give me some advice, cuz I cannot stand arguing every night when we come home.