Shitty boyfriend? In love and stuck

Im 19 almost 20 and just had a baby boy a month ago. Ive been with my boyfriend since i was a freshman in high school and am deeply in love with him still. The first year was great but then he started getting physically and emotionally abusive. Since the beginning of my pregnancy he has been a lot better other than one time pushing me into a fence. The thing is he doesn’t work, sells weed for a living, and now is mad at me for his brother molesting me. He feels I’m trying to keep them apart. I don’t get how he thinks sometimes. I fear my love for him will keep me with him when i feel i should leave. But if i leave i fear my life :/. Im not sure what to do and could really use peoples opinions. Sorry if i sound all over the place. Need support. :(