He didn't drink BECAUSE you drank around him (though you do realize that is enabling, right?). He's right about one thing, though. HE has no problem with his drinking--YOU do.
So, given the fact that you can do nothing about him, what are you doing about YOUR problem?
I'm sorry--ruined vacations suck.