boyfriend drug rant - advice please

Becky
So this is gonna be a bit controversial. I think. 
So basically I got pregnant really quickly with my boyfriend - I'm currently 25 weeks. When I started seeing my boyfriend he told me that he had issues with cocaine but he was clean atm, fast forward a month and a half he started using again but only socially so I 'allowed' it (regretfully). Since we found out I was pregnant he's been stressed and has been doing it 1-3 times a week, blowing basically all of his money weekly. I gave him chance after chance to cut it down and get help, and every time he just carried on. And last Sunday I was done, I left his and came back to my mums and basically gave him the ultimatum of 'touch it again and we're over' he was sorry and all that. But I've made a point of not seeing him all week as a mini break from each other. Anyway he was supposed to come to mine tonight but didn't show and I sucpect that he's 'on it' for various different reasons, which could also (unlikely) be innocent. 
I have also said that if he carries on taking it he will not be allowed unsupervised visits with his baby once he's born - which I think is fair?! 
Point of this point is me being nervous and anxious that he has done it as I've given him ultimatum and it will mean I have to leave him (or he will think I'm just all talk) and I'm terrified of being a single mum at 22 before my baby is born. 
Rant over - any (constructive) advice?