I'm no doctor, but I speak from experience. I bet your anxiety is due to your drinking - that is, withdrawal. You think it helps, but it actually makes it worse.
I had a period in life when I was constantly anxious...life was hell! I'd drink at night and then in the day I would feel anxious until I could drink again at night. It's a cycle. I had always thought my alcohol intake was pretty normal - I was rarely getting blitzed, but I was still drinking to excess. It wasn't normal.
It took a few weeks after making my changes, but I noticed that the anxiety went away. Of course, I have stress...everyone does, but it's much easier.