You're expecting logical, reasonable behavior from an addict.
You're taking his absurd comments to heart and allowing them to hurt you.
These are classic deflection tactics, as you seem to see on some level, but you are allowing them to get to you and cause you to question yourself.
Have you been to Al-Anon? Talked to a lawyer?