I think if alcohol is negatively impacting your life then it is too much.
I know that I found this site while googling concerns about my drinking. If you found it the same way, that is a sign that your alcohol use is probably causing some sort of problems in your life.
I spent three years alternating between periods of sobriety, and periods when I attempted and failed to moderate my drinking. I finally said enough, and have been sober for a just over a year now. I am so much happier in all aspects of my life, and even the difficult days are easier to handle.