Nephew hitting my newborn

I don’t know what to do. I love my nephew and I’m closed to his parents. We meet up a couple of times a week. The last two times they have come over my nephew has come out of nowhere and has hit my newborn hard. We share a house with my parents in law. Today I went to the store and when I came back they were here. My nephew threw an object at my daughter and it freaked me out. I skipped dinner and took my newborn to the room with me. I feel so bad but I don’t want him around her cuz I am afraid. I am also worried that this is going to ruin my relationship with my brother in law and his wife. I don’t know what to do.