Newborn vs Toddler

I just had a newborn and I have a toddler. I thought my newborn would be difficult to deal with but no I'm loving it so far but my toddler annoys me. Everything he does ANNOYS me. I don't like dealing with him anymore. He is constantly doing something he isn't suppose too. Im constantly yelling stop or no at him. I feel bad. He isn't getting any less attention because I still do one on one activities with him.