You should be able to get a restraining order, based on the verbal abuse. That would get him out of your shed, and give the police a basis for arresting him if he comes back.
I'm sorry you're going through this, but I agree with the others that having him away from your home and the kids is best for everyone.