I give up

So I have been hoping and praying for my boyfriend and I to have a baby together. Well the other night he told me that a doctor told him good luck on having any more kids being as he supposedly has a daughter and son. He wasn't supposed to be able to have kids in the first place. I'm just upset because I have been going on hopes and dreams and now we will more than likely NEVER have kids together. What do I do? Is there anything I can try?