How to handle same age siblings hitting your child?

I am 27 weeks. I don't live with my boyfriend, and don't stay the night with him ever because he has anger issues that I found out about during my pregnancy. I live with my parents right now. I have siblings in the same age range as my child where they will be playing with my child. The ages range from 6 months, almost 2, and 3 years. My boyfriend said that he's worried that my brothers and sisters will hit, bite, ect our child. It will happen because that's what children at least once and hopefully at most once. It's not like our child will never hit my brothers and sisters. He said "I will hit the shit out of them if they hit my child." I told him no, you can't do that because it's not your child and you shouldn't hit the shit out of any child. In our house we only spank sometimes (barely ever) but send the kids to their rooms more so they can actually think about what they did. He then told me that if nothing gets done he will "go the fuck off" and take our child out of the house... which is fine because I'd rather him handle it that way than to hit another child. I became very disturbed that he said what he said and haven't talked to him since. What's the best way to handle it in your eyes?