Why don't you take the remaining money out of checking at least and put it in your own account?
That way you can maybe get by a bit longer if you aren't ready to do anything about asking him to leave.
Another option might be to leave yourself, take the kids, and go stay with your parents for awhile.
My take on this is that you are allowing him to take the money and there is zero incentive for him to stop drinking since you are putting up with it.
I know it is hard, but the kids are being hurt anyway by being in this situation.
They absolutely feel the stress, anger, and tension much more than many adults realize.
I was one of those kids and it was very harmful growing up in an active alcoholic's home.
I'm sorry for your situation but it does sound like you need to take action and soon.