You have to consider the source. Your husband is not the best judge of anything right now. IF he is working his program, he will eventually see things more clearly--but they will be things about himself, not about you.
Keep up your Al-Anon. As TG said, you each have your own side of the street to keep clean.
Next time he says something like that, try smiling and saying, "We're both works in progress." That's it. You don't need to defend or explain anything. You don't need to take what he says as true. You don't need to argue the point with him.
And it's true, IF you both continue with recovery, you will both grow and change for the better. You will both be happier people, individually if not together.