I need to vent ☹️ not my baby and still want one

So this month has been absolutely crazy for me. My husband and I have been together for 15 years, married for five (he is 30 and I am 27). We have been trying for a baby for one full year now. All three of my sisters, have two or more kids and don't seem to have trouble reproducing. I will also mention, my three sisters have had serious drug problems (along with my mom) and have had a hard time keeping their lives and kids together. I don't associate too much with them because of this. My youngest sister (22 years old) just gave birth to a beautiful baby boy in July. The problem was, she had drugs in her system and the baby went through withdrawal. Children's youth services had stepped in and took custody. When I found out, my husband and I agreed to fight for custody to give this child a chance to be a normal baby boy until my sister gets her act together. We got him at 3 weeks old straight from the hospital after we won custody in court. He is now 6 weeks old. He is healthy and happy. We love him very much. It isn't easy because I don't get maternity leave, or even 9 months to mentally or physically prepare for a baby, but we are making it work while working full time. I am a family medicine  ractitioner in an urgent care and my husband is a union carpenter so we make pretty good money and can afford to give this baby a good life. I am hoping I am doing the right thing, but I feel like this sets us back from creating our own family. My other problem is that I've become attached to this little guy and I'm afraid that I will be heart broken when it's time to give him back. I just needed to vent and hoping for some positive answers and feedback. Thank you for reading!