I found that my anxiety was higher during the 5 days of acute withdrawal but I've found that since then, my baseline anxiety is vastly lower than my former hangover anxiety. I've also found that my baseline anxiety sober is substantially lower than my baseline anxiety when I regularly drank. My anxiety isn't all gone, all of the time, though I'd like it to be, but I know that alcohol will only magnify what's already there.
I also realize in retrospect that when I thought "a little" alcohol made me "smooth" in social situations, it was obvious to all that I was drunk. Social situations are like unheated swimming pools: just dive in and you'll quickly get over the stress.