I was suffering crippling anxiety during my worst drinking period. I could barely function at work and the only relief was more booze. I couldn't wait until 5pm each day so I could get respite from my thoughts. It took a month of sobriety for the anxiety to go away which is why it was so hard sober up.
Sounds like you are in a good place now. Don't ruin it by thinking you can have a few,