Adopting

My husband and I have been together for almost 6 years and I am going to adopt his son from a previous marriage because the mother lost her rights a couple years ago. Well we found out a year ago that her other child was placed in foster care and we got our foster licence and did all the paper work to foster him but both the case workers we had sucked so bad and didn't turn the paper work in time so we ended up not getting him then. Well last week we got a call from the agency and she and the child's father lost there rights completely. So we are trying to get him now. The agency is saying though that we are not family which my son is his half brother and the reason they don't know about each other is because she lost her rights and the reason for that is because she stopped coming to the supervised visits. So how would they ever be able to meet and why would I put my child through all of that again? Any advice on how to go about this would be great. Oh d they told us they there are 2 other families willing to adopt him but again why wouldn't you want him with family.