father knew I was raped

When I was 14, my brother was raping me. He was 27 at the time. I wasn't the first girl he did it to. My father knew he was doing this and still covered for him. I am now 18 and have not seen any of my dads side of the family since because my dad made them all take his side. I'm married and have two daughters. One is a year old and the other passed away not long ago. My husband and I have moved out of our house and back into my fathers house for a few months so we can get back on our feet after losing our daughter. 
I feel really awkward here, but we have no where else to go. My dad talks about my brother all the time. Trying to get my husband to work with him, showing me pictures of his kids, he talks to him about me all the time. After losing our baby he called and said "your brother is sorry and wants to know if he can help" 
Now.. I have a daughter whom is one like I said. I'm terrified to allow my father around her alone because he knew I was being raped and tried to hide it. He randomly just comes up and takes her from my arms and takes her to a different room to play and I just don't know what to do. I cannot talk to him about it because he just makes jokes about it and we really need somewhere to say. He also cannot take anything serious. He has a girlfriend that allows my brother around her grandkids all 10 and under. She knows what happened. I just don't know what to do.. help?