I want to commend you, getting your children out of that toxic enviorment is so positive!
They sound like they are having a ball.
As for him leaving the house, I would tt to your attorney, they know all the ins and outs of how to legally handle that issue.
We all have little setbacks, you will be fine, keep moving forward.