I don't know if you still go to court and if you need to set ground rules through the court about what is acceptable and isn't. You could set up formal agreements that he must x,y,z (be on time, give you 24 notice if he cancels, be sober when driving the kids, etc) as a way to have structure.
I don't know anything about the law, but there must be a legal way to set responsible, safe boundaries with him that have clear consequences.
Maybe talk to a lawyer?
Hugs,
peace