IMO, a loving and respectful spouse would stop his extra-marital relationship with the other woman simply because you indicated to him that it is hurting you and/or causing problems for your relationship.
IMO, the fact that his excuse for being unloving and disrespectful towards you and your needs, all in the name of 12-Steps, Recovery, and goodwill for his friend or whatever, is total and complete bull$hit.
IMO, it does not matter what he thinks your own "issues" are or whether or not you are issuing an ultimatum, and these accusations of his against you are just more of the same kind of BS alcoholics and addicts throw at you in defense of their own questionable behavior.
IMO, in order for there to be trust in a romantic relationship such as between a husband and wife, each person must care about how the other perceives their behavior. It does not sound like you trust your husband, and it does not sound like your husband gives a rat's a$$ that you do not trust him. I am sorry you have a husband like this.