As long as he's not tearing up your turnips in a drunken rage Thanks Notmyrealname, that made me laugh so much. There's a 'yet' we all need to be mindful of!!
Gibbons2, to your question; for now I'd continue to cultivate the veg patch, assuming he joins you there to tend the veg soberly. It's neutral territory, you can touch base and keep contact if that's what you want to do, without entering into his drinking or your sobriety. As others have suggested, if it were me, I too would focus greater energy on cultivating new friendships, and establishing a sober lifestyle for myself.
Wish you well