3 year old lashing out

He's not responding well to his baby brother still who is a month old. He's been not listening, hitting, kicking me, and he will hit his head into the wall when I tell him "no" and say "ow" so I feel bad for him. He is great with his baby brother; not once been aggressive toward him but he seems to always be mad at me until later at night he wants to cuddle with me. He is a bit delayed and he's in speech therapy so I know some things get really frustrating for him to convey and I know he's upset I don't have as much time like I used to because of work and my other son/the new baby, but because of his speech delay he can't understand me explaining how much I love him. That if he stopped being mean I'd take him out for him and mommy time. But he is seriously so mean to me lately I can't get through to him. I've tried signing to him in the signs he does know, I cuddle at night, etc. I'm really torn up over this :( his therapist said he'll grow out of it but it's hurting me a lot and i even began taking antidepressants.