Tough situation, for sure, but as I see it you have two choices. One is to continue as you have been and work on detaching from his unpleasant attitude. How is he with your son? Is he patient, kind?
The other, of course, is to get a firm parenting time schedule in place, with some kind of sobriety monitoring. Given his history of "slips" and relapses, my bet is that the court would go along with that.
I just don't see how you can have it both ways. You can't make him be nice or considerate to you.
Wish I had a better suggestion.