Use brother for insemination

After all the tests it is looking more than likely that my husband is totally infertile. We are exhausting all options including cutting him open to find sperm.  However, as a back-up plan he wants to use his brother's sperm to impregnate me.  We talked to his brother and wife (who do not have children of their own because 2nd marriage for both and she is too old). They are happy to do it. Weirdest conversation ever!!! I am not sure how I feel about having his brother's baby. Love idea of getting at least some of husband's genes. Nervous about weird family dynamics.  What do you all think?