Cat help!!

So we bought a cat from the shelter. He seemed sweet. We have 2 kids. Well my 2 year old was at her grandma's. Just me my hubby and my 6 year old were home.. Cat was using his box and OK the first few days. As soon as my 2 year old got home, things changed. He hissed at her, just didn't like her. He peed on her stuff, pooped all over our house. I tossed him outside in pure frustration. He came back tonight and is crying on my patio. I put food and water out there. But he is not coming back in. Should I take him back to the shelter? Or keep him as an outdoor cat?

Update ive decided to give him back to the shelter and let then know to inform whoever else adopts him DO NOT PUT HIM IN A HOUSE WITH SMALL KIDS. I didn't just throw him out overnight I've had him for 2 weeks and he's done this. And I even paid to have him bathed and groomed. Didn't have any intention of not keeping him. But I can see he just has behavioral issues towards my daughter that I will not be able to tolerate. My child is more important than a cat. And my cat before him RIP.. Was a shelter cat and never acted this way. Thanks for the help.