Baby or not??

When my husband and I first started getting serious he always told me he would give me another baby, we both have children from previous marriages. Now that we are married, have purchased house (paid in full), really the only bills we your basic bills. My family has asked on numerous occasions when they would get another grandbaby and every time I try to bring it up to him he dodges the question, leaves the room, pretty much anything to avoid the conversation. Any suggestions on what to do???