This might seem like an obvious question.. but why do you answer his calls?
I think GL is right, and you seem to know it too.. he's an addiction to you. But seems an UNSAFE addiction. You're on the train by choice (crazy train), maybe it's time to get off? Set some boundaries.. maybe don't talk to him if he's been drinking. Don't answer the door if he's drunk. You can choose to end some of this..