How very alarming. I would speak to an attorney for guidance. Does he push for visitation? Could your children set the terms so to speak. For example, dad, mom is going to drop us off at the movies and will pick us up after, lets go see X movie? Its just a thought. Thats sort of how i hsndled things when i first separated. I also made sure my children have cell phones at all times. Ideally if the police had been called right then that would be that. I made it clear to my x if he ever drives my children under the influence i will call the police en route imrdiately. He knows i mean it, and i do.
I personally would tell him i plan to call the police immediately and he will likely back off visitation. I also encourage the truth, the children are not safe w you and will not be getting in a car w you.