Husband's pedophile brother

My husband has an adoptive brother who is a pedophile. A non offending pedophile but he opened up to my husband about it. He was adopted from the foster system. My husband's adoptive brother does try to suppress those urges and he's on medication that basically takes away his sex drive and lives in a secluded area in the woods. He doesn't go around children. He doesn't go to family events. He has a therapist he sees twice a week. I understand he's trying so hard but I won't lie when I say it makes me uncomfortable that my husband speaks to him. My husband calls him once a month to check up and goes out to see him bi monthly. My husband also hasn't told anyone about his brothers feeling about children because he doesn't want to villianize his brother when he brother when he hasn't done anything yet but has said if allegations against his brother ever come out or he goes around children then he will say something. Him and his broydo have boundaries. He's not allowed to see pictures of our children. He's not allowed to be friends with us on Facebook. His brother doesn't even have Facebook anymore. He keeps his brother at a decent length but still close enough to show he cares. Is it wrong that I still have uneasy feelings?