Worried about interacting with Babies' Father

Long post.

I was in a relationship with the father of my children for 8 years. We have a 2 year old and a 8 month old, both boys. He is an alcoholic and became more violent toward me and our 2 year old while I was pregnant with my youngest every weekend we would stay at his house. Each time there was an incident, my 2 year old was in the room or right beside me. DCS became involved and he gave up his custodial rights. I have full custody of both Babies and was told that we could communicate and figure out how to do visitation. Now that court is over and the no contact order and everything has been lifted, his friends and family members have contacted me saying that he is still drinking and that nothing has changed. he calls me constantly and says things like he wants to be a family again and can't wait to see the Babies. But when we were on the phone my 2 year old heard his voice and asked if it was his daddy but he wouldn't acknowledge that he wanted to talk to him or anything. He only asked about me. Before all of this all he did was give me money for our first, they got on to him all the time for every little thing he did and sometimes he would push him on the swings at the park. That's all. He wants me to bring them to their grandmother's house so we can hangout and they watch then for us. But I am nervous. I don't trust him or his family enough to let them be alone with either of the Babies without me right there with them. I don't know if I can handle going into that house again because a lot of bad experiences happened there.

Part of me wants to be a Happy family and I want to hope that we can be again one day but I also know how he really is and that I have to do the right thing for my Children and keep them safe, happy, and healthy. I'm trying not to go back into what it was before. I'm not sure how to handle visitation. I don't know if it would be good for them or not.