Number of kids

So to make a long story short, before my hubby and I got married, we both agreed we wanted two kids. Well fast forward a year in to our marriage, he says “I changed my mind. I only want one kid now.” Of course I’m like whoa what changed because I’ve ALWAYS wanted two kids. He responds with “Well my mind just changed I guess. Am I not allowed to change my mind?”. So whatever I let it go. Now we are pregnant with our first and every once and a while I will say when they have a brother/sister. His response is always “don’t get your hopes on another one”. I guess I’m just wondering has anyone’s husband also said this, but then after having one, changed their mind back again? I’m just looking for some hope because I really don’t know how to feel.