I don't trust him

My ex and I broke up after months of being told to either abort or adopt my baby out once it's born. I know now that I'm having a little boy and I couldn't be more excited. I've been loving on him since I found out he was there - I want to raise my son and have a good life with him. He is the most important thing in my life. 
That being said, suddenly my ex wants everything to do with him. Ultrasounds, buying me a pregnancy pillow, and OH he decided to see a lawyer and wants 50/50 custody of this little darling who he was so adamantly against having in the first place. 
I get that guys can change their mind and come around to seeing a baby as a good thing, but I've gotten my mind into a setting of "I can do this by myself" and have been looking forward to a future of just me and my lil guy. 
That being said I honestly don't trust my ex with my son. I don't think he would do something harmful on purpose, but I know he's never so much as handled a baby before. He doesn't know how to change a diaper. In fact he always made faces and comments when we drove past children saying how their parents were part of the problem overpopulating the world and stuff. He's always openly hated children. Even stopped talking to a friend of his because she was pregnant. I just don't know what to do. I know I can't refuse him his rights as a father to know the child, and I would never do that to a kid, but the thought of him having this child half the time - this child that I have loved and fought for? Who I KNOW will be worth every morning noon and night of throwing up, restlessness, back ache, and leg cramp... It honestly offends me a little. He and his parents just wanted to get rid of my baby and now he wants to take him away from me half the time?