I'm no expert on this stuff, but it would seem to me that your being late and his drinking are really two separate issues.
If he's angry at you for being late, there are other ways to confront that issue. If he chooses to drink because he's angry with you, that is not your fault. At least, that's the way I see it.