Am I causing another speech delay?

I have a 2.5 year old. I feel like I did him wrong because he still isn’t talking. No known speech delays in either sides of my or dads family. He has said words but it’s a one time thing and doesn’t happen again. He is very verbal but no words. He is in speech therapy. I am teaching him sign language so he can communicate somehow. I feel depressed about it because I hear kids his age and younger talking and speech therapy doesn’t seem to be helping :( I also have a 15 month old. He is also verbal but not saying words yet. He is picking up on sign language though. He signs 3 words. Which isn’t a lot I know, but I’m worried with him picking up the sign language he won’t actually talk either. Can I get someone’s opinion if I’m doing the wrong thing by signing around my youngest when talking to my oldest? :(

Edit:

I do say the words when signing as well, forgot to add that!