Blame shifting is a common ploy amongst addicts. You mention his drinking and he brings up something you have done that has no relevance, and puts you on the defensive.
Many people have described their alcoholic as having an "affair" with alcohol.
I'm sorry he said things and hurt you. Its so common. Its a straw man's argument. My husband would only have to tell me to an an affair once. You ask you get. Not really but it would take me a lot to get over that and I am not sure I would.