What should I do?

So today I found out my ex who is my unborn babies dad hangs out with certain guys. These guys are into everything, pills, coccain, drug dealing, a lot of violence. And they have even offered him drugs. He told me this himself and he told me he refused them. But I just can't trust him. While we were arguing about it I said I don't want any involvement with him and I don't want him being involved with the baby while he associates with people like that. Before this argument he has been involved in everything in the pregnancy and been at every scan we were even planning on getting back together (which is out the window now). He's the type to say yes to things just to fit in with the crowd and I absolutely do not believe he doesn't do drugs with them. Obviously I can't prove he's doing it. I don't want to deny my son a father if he actually isn't doing it. So I don't know how to handle this situation. Do I wait until baby is born and let professionals arrange something so that he's not under the influence of drugs when seeing his child? Or do I carry on letting him being involved and make him do drugs tests regularly? (Which he offered to do btw). or do I just forget all that and trust him? Its hard because i do want him to be involved with his son but i don't trust him. Honestly I don't know what I should do , what would you do?