You need to move quickly. Get an attorney who is DILIGENT (I hear they do exist, can't confirm). You need to file something ASAP to show that you are not withholding visitation for any reason other than that you are trying to protect your children. I hope your communications are via text? Always use written communication. If at all possible, choose email- courts prefer it. But I would not speak to his mother by phone if I were you. She may deny the conversation.
Sadly, many lawyers will tell you to abide by the court order until it is changed. And judges can be very lenient with addicts. I'm sure my judge would say, "Well, what was the problem? The mother did the right thing. She removed the children from the situation." Then my ex's attorney would go on his "She's a parental alienator out to get my client" shtick.
It's a mess. Family court is a MESS! But do what you feel is right, no matter what. Personally, I am so glad to have professional supervised visitation. Our monitor is an absolute idiot who sings ex's praises (ex missed FIVE straight months of visitation and this was barely mentioned in monitor's report), but I still can't imagine having one of his flying monkeys do the job.
Good luck, hang in there, stay strong!