Anxiety is uncomfortable for a while, maybe only a few minutes, but it isn't fatal. Drinking, on the other hand, can most certainly do you in.
Can you stop? Of course you can. If you have medical concerns, check in with a doctor, then make a stand and get to it.