Hey Christin ... don't feel bad about not sharing your child with him. He doesn't deserve that child right now. He is abusive and has made the child a nervous wreck. You are on the right track in protecting your child from him.
Your husband vowed to love, honor, and cherish you. That includes the kids. He has not lived up to his vows and now will pay the price.
As you well know, your children only have one responsible parent - that's you. You are the provider. You are the protector. You show them love. You are doing what responsible mothers do - protecting your children like a lioness protects her cubs.
So your AH is weepy. Boo effin hoo. He should grow the hell up and be responsible. I have no doubt that you gave him a million chances. Now that the jig is up he wants you all back. Well know this fact - what he wants is for the song to remain the same ... he plays the tune and you all dance. Enough of being his trained monkey already. He is an alcoholic and active acloholics know nothing but their own need to feed the beast.
You also must realize that you might save his life too by no longer enabling him.
This is in your capable hands. You do no one any favors by folding, not even him.
Be strong.