So, for now - decided to take the kids to my sisters and leave them there for their summer vacation (after their school ends in two weeks), I come back alone and figure out a solution to making him leave the house. And either start divorce process or pack up and go back to my sister's.
it might be prudent to run all that by a lawyer, as you are taking the kids out of state and leaving them there, without either parent. i'm not sure that is legal if the father is not in agreement and you are still married.
it could be viewed as abandonment by you in court.
everyone here understands it is not abandonment, it is leaving them with a loving adult to get them out of a toxic environment, but if he wants to get ugly with a good lawyer, it might not be within your legal rights to do so.
if you go to your sisters with the children, what would you do for work? can you work remotely from there?