Deciding how many kids to have?

How did you know you were done having kids? We have two kids now. While I was pregnant with my second my husband and I said two was enough and he agreed to get a vasectomy but has not scheduled his appointment. I thought for sure I didn’t want anymore. My husband says he’ll give me as many as I want. We could afford it. But we only have a three bedroom house and so someone would also have to share. Buying a four bedroom home in our area is out of the question. Sometimes I think of how nice it would be to have a real big family. How did you decide?