Spouse should support and build each other up.
If he is dragging you down don't listen to him. Easier said than done but like the others have said no one gets to determine your self worth but you.
Stay sober and block out his negativity. Focus on the good inside of you that you know is there.
We have ALL done things that have hurt others but that doesn't give them the right to beat us down about it forever.