I agree with FourOurGirls. The longer you wait, the harder it gets to make the case that the issue is serious. It sounds like you have some control of the situation; maintain that control by getting a court order in place that keeps him from being able to drive the kids and/or do anything else detrimental around them.