Need advice on my ex husband

Hi ladies, I've never posted before, but have seen how supportive this community can be and I could use some input. 
Back story: Divorced my heroin addict, controlling husband and I have full custody of our 2 boys. I did give him the right to see them on the weekends, but have been avoiding it due to him living on the streets. Every now and then he will stay with his mother and so the I let the boys go hang out with him, even on weekdays. Especially now since its their summer vacation. I've never wanted to keep them from their father, but I also didn't want them to have to see him so broken. Anyways, he won't contact them for weeks at a  time then when he does, he wants everyone to drop everything so he can see them. So today he had his dad pick them up and bring them back to his moms. Everything  was fine until he wanted them to spend the night, which I wasn't opposed to. I'm not a fan of his mother, but that doesn't mean by boys have to avoid their family because I don't like her. That's another story for another time. I was simply having a conversation with my 14 yr old via text and then his dad took his phone. Here are the texts. Please tell me if I was out of line for saying what I did to my son. And if you have any input on to handle these kind of situations.