I think you're right. The urge to defend myself is strong. The urge to give some response so I can't be accused of ignoring his 'effort' is strong. It's frustrating to know that I'll continue to be accused of kicking my daughter out of the house when I didn't. And yet I know no answer will do any good.
I've considered reminding him I already told him why I walked away and he already told me none it happened, and telling him to please go to AA and then maybe we can talk.