Just curious but how long have you been drinking? I had the same issue you did in the beginning - drinking causing panic attacks. I then ended up on medication for panic compounding my intake of chemicals.
What I'm thinking is that maybe like the others have posted here, it is that you are progressing into alcoholism - it is a progressive disease. But I don't claim to know much. The panic attacks should have been a warning sign for me - I ignored. In fact I started to drink more to hold the panic at bay.
I agree with the other posters who say it doesn't matter how much you drink it what it does to you. The panic could be a warning sign.