When your BFFs son is aggressive to your daughter

M
Ok so long story short, my BFF and I have our kids about 5 months apart. He is a little over 2 and mine is just under two. Well we were over their house playing and he kept hitting my daughter in the face, pushing her, grabbing her shirt and trying to scratch her face. He goes to school but so does my little girl and she isn't aggressive. I know his parents aren't aggressive bc they are wonderful people. Maybe this is just behavior learned from school? I felt terrible for my daughter bc she has NEVER had another kid hurt her intentionally like that. 
If she acts out physically for me, she gets a time out for about 10 seconds and the. It's finished and over. My friend just kept saying "no" and be nice" I swear I wanted to put him in timeout myself. I never said anything and Charles it up to maybe he was frustrated and he was acting out that way, but on the way home I decided I can't bring her there till he starts getting disciplined better rather than as calm taking to. I mean he was intentionally scratching my daughter's face. I was trying to keep him away but I also have a 4 month old and didn't want to make his mom feel bad bc she just had another baby herself. I guess my question is, what would you have done!? How dO you tell another parent that they need to step it up or we aren't coming over!?!
 My mommy guilt was terrible coming home  bc I felt like I hadn't protected my little one enough!? Did I react ok or was I being too much of a push over to not hurt my friends feelings?  What would you have done?