You can TELL him to leave, but if he chooses not to, your only option really is to leave, yourself. I agree that since you have the kids it makes more sense for you to stay put and for him to figure something out.
You might want to talk to a lawyer if you are considering any kind of long-term separation. Depending on where you live, you might be able to file for a legal separation and get an order giving you temporary possession of the house. If he's the type would would show up at the house ("because it's my damn house, too") that would give you the legal right to change the locks and for him to be charged with trespassing if he came in without permission.
Mom's house sounds like a reasonable short-term solution for you if he refuses to leave.