Second baby?

My husband and I have a little boy who will be 2 in a couple months. We are doing OK financially right now but have very little wiggle room. We always figured we would have more than one child but I don't want them to be more than 4-5 years apart in age. My brother and I are 7 years apart and didn't get along well as kids. My husband still wants another baby. I am not sure what I want. On one hand I would love to have a little girl but on the other I am not sure it is responsible of us to have another now. My husband gets upset when I say "if we have another" even though we have talked about it. If we got pregnant we would do what needs done to support the baby but I am afraid to have sex for something to go wrong and get pregnant. I don't know what to do.