Opinions appreciated

My husband and I have been married for close to 7 years. I have been blessed with one spoiled little guy who is happy, healthy, very energetic!  I have always wanted a big family.. My husband no more than two!  I was ok with two!  My husband is the love of my life, we keep God first and never argue! He is absolutely in love with me and would do anything for me! My son is 4 (5 in January). I have been wanting another baby, but my husband says right now we live comfortably and can spoil him.  He grew up semi poor with 1 sibling.  He does not want our children/child to grow up wanting or needing something and us not being able to afford it... He thinks a second child would take away from our son now.  Plus he says he don't see how he could love another child as much as he loves our child now.  I have been talking to him for the last couple of years about having #2.  His response was we are so blessed and happy with one.  He did not want another one. Well a few months ago he said he was thinking and he said if I wanted another child... Then let's do it!  His reasoning... He is afraid that I would eventually resent him if we only had one, and he wants me happy.  
I have mixed emotions about this... At first I was happy..stopped taking birth control.... Then when I thought longer about it... I told him this month I was going to get back on my bc.  I wanted us to both be on the same page, not just to make me happy!  
It hurt his feelings... He was so upset that I was going to get back on bc.
I have not gotten back on them ... I want to grow our little family and I want him to want to grow it too!!