I need to vent!!

Me and my boyfriend have been together for a little bit more than a year. When we met, he told me he had three daughters. I have a one year old. After living together, he told me he had just found out he had a 3 year old boy. I didn’t mind of course that was way before me. But now this kid has been around for the past weekends and he is wild! He pushes my one year old to the floor he starts to kick him and my boy cry’s. I told him to stop doing it that and my boyfriend told me I had no right no scold him or tell him anything. Sooo... my boyfriend doesn’t tell him anything either he doesn’t tell him to stop being aggressive. Last night he mentioned on how my boy is so sensitive and cries for everything and that he needs to learn to defend himself. He just turned one in October. His kid is about to be 4 so there’s a huge difference in age. This is starting to get so frustrating and stressing. I’m 20 weeks pregnant and this is just too much for me.