If he's back to drinking, I would make a case for not leaving her with him. At 4, she's not able to call you, get out of the house if there's a fire, get help if he passes out, etc.
And I'm not saying "you're a bad mom for leaving your kid with a drunk." I know how sometimes we have no good choices. I've been there. But I guess what I DID want to tell you is -- there is nothing WRONG with you keeping your daughter from him when he's drinking. There's no quality parenting or interaction going on between a child and a drunk. Or between anyone and a drunk, really.