When I first (consciously) started doing boundary work, I realized that I had been (unknowingly) TRYING to set boundaries with people in my life but that I had gone about it in the wrong way, that is, by stating my preferences or my feelings to them (usually through tears) and hoping they would honor and protect me. I would say something like, "I don't want to be around you when you are drunk," or "When you do that it makes me uncomfortable." Or the usual, "When you do that, it reeeeeeeeally hurts my feelings."
Of course another person is never going to be able to meet all of my expectations, desires, or wishes and so I always wound up getting hurt. In most cases I'd say it hurt more that they BROKE the boundary than the actual boundary-crossing itself. That's because I would always interpret the behavior as, "He doesn't care about me..." or "He doesn't really love me...otherwise he would not have done that." Most of the time, though, I never verbalized what I preferred (what my boundaries were) and just would get hurt and hysterical reacting to everyone else's behavior.
So, now that I am better-practiced at boundary-setting and communicating, I see that, in large part, boundaries are less about the specific things I want my partner (or whomever) to do or not do, and more about the way I think, every day of my life. The biggest difference between then and now is that now I: Take 100% of the responsibility for my life; I know what I want and what I don't want; and I know what is acceptable and unacceptable in the behavior of others in my life. I have made MYSELF responsible for my own feelings and for maintaining my own peace and serenity. I hand these things over to NO ONE now. EVERY day I take responsibility for who I will let in and who I will not (getting better at this as I age and learn). These are the KEYS to my boundaries.
The setting of INDIVIDUAL, specific boundaries such as the "When you come home drunk it upsets me...therefore if you come home drunk again I will leave you" kind, are more an OUTCROPPING of my taking full responsibility for me. Funny thing is, though, I did not become strong in my boundaries until I practiced one of the more specific boundaries with someone who I had known for a LONG time, and who I KNEW loved me even though his alcoholic behavior indicated otherwise.
Sounds like this guy is someone you also have known for a long time; perhaps someone "safe" to practice individual boundary-setting with. Give it a try. Set the parameters for your relationship with this guy, and see what happens. Just the act of DOING it will show you just how strong you really are.