If your boundary is that you do not talk on the phone when he is drinking you simply make the statement you've thought of ahead of time for such a situation and hang up. No need to ever discuss that encounter again. If he brings it up simply state "I won't speak with you when you are drinking." and say nothing more about it. Walk away if you must.
If your boundary is that you are not around him while he is drinking, then when you get to his house and discover he has been drinking you say "I gotta go. I'll be back when you aren't drinking.' and turn around and walk out that door. Every.Single.Time. You don't have to worry about an argument because there is no discussion. You simply leave without saying anything further.
I struggle mightily with boundaries but I have learned one small thing - don't discuss. The discussion is a hook, a manipulation, an invitation to trample my boundary, my self respect, my very sanity in the end when my thinking was so cloudy and confused. Say what you mean in as few words as possible and then be quiet and leave. That is the most powerful thing ever, and it is something I can do. It works especially well for me if I have a few one liners tucked in my pocket because they come out without having to think on my feet. They keep it focused. My xah went nuts when I refused to engage/discuss. He subconsciously knew that if I quit the discussions, he lost all his power through emotional manipulation, which he was very very skilled at. This caused me anxiety because I have a lot of anxiety when other people are uncomfortable, especially if I caused it, but I was mostly grateful that it was him feeling all the anxiety and desperation and not me for a change. Also, I couldn't get away from him because we lived together. I had to just ignore the endless stream of things he'd say to me. He'd blab on for hours and I wouldn't say 10 words. I couldn't leave because I wouldn't leave the kids alone with him in the end. I had no physical boundary until he moved out. My silence saved me I think.