Feeling pretty bummed

So a little backstory. I had my son via c section because he was breached. We just planned a c section for my second at her due date but trying for a vbac. Today I discussed with my doctor having kids in the future. She suggested not having more than three c sections because the risks for complications go up significantly after that. She is very pro vbac and only scheduled the c section because my daughter is already weighing in at almost nine pounds at 37 weeks and I have a small frame and I am comfortable with that decision. I am very sad because we want a large family of five kids and it looks like that may not be in the cards for us. I know many people have had more but my husband is very concerned about the risks involved after three. Anybody else been through this?