I drank large amounts of coffee during my drinking years. I gave up alcohol recently mainly due to anxiety and severe panic attacks. I decided to cut back on the coffee at the same time because I was drinking a lot of it - maybe 1.5 liters a day.
I drink virtually no coffee now.
Very occasionally, if someone offers me one, I will drink a coffee. Almost every time it will lead to a panic attack. Not a bad panic attack, but its still unpleasant.
So for me now, coffee and anxiety are related.
But I am fairly sure, if I didn't have a problem with alcohol in the past, then coffee would be fine for me now.