Looking for advice on a situation

Hi everyone so a few days ago one of my closest guy friends told me that he was going to be a dad (I would be godmother). The pregnancy is the result of a one night stand he had so it was not planned but he is excited nonetheless. I am obviously happy for him. He questioned when my husband and I were going to have a baby. No one knows that my husband and I have been trying for a while so I informed him that we haven’t been having any luck. He made a comment/joke about how it hadn’t happened yet since he managed to get a girl pregnant the first time without trying to. He’s the first person I opened up to about our struggles so this response obviously hurt my feeling and I didn’t know how to respond. I am currently in another tww and struggling to stay positive. Any advice on how to handle the situation with my friend? Any advice on how to respond to people asking all the time about when we are going to have a child?

Thank you!