I'm still waiting for the part of the conversation that went well. Reading this has my head spinning so much, I felt like I was reading my own journal. I would journal and then read it when my head would stop spinning, to see what, if anything, was accomplished, and why did I feel like a train just ran over me by the end of the conversation. But, I found out, I was hit by that train.
He clearly spun everything back to you. I can see him clearly having a drink, and you look at him, and he shoots back at you, so I guess "I'm an adulterer".
My ex pulled the "rape" thing on me also, but for a different reason. Mine wanted me to agree with him that if I said "no" to sex, it wouldn't be considered rape, if he kept bothering me until I just gave in.