baby daddy issues

my baby daddy is a drug addict. he had been sober for about two months when i got pregnant but as soon as i found out and told him he left and got back on the drugs. now i’m wondering if i’m wrong for not allowing him to be around or even see the baby??