I tried cutting down for years, with no luck, and I drank a lot less than you and a lot less often. I never had physical symptoms of withdrawal either. It isn't so much how often you drink or how much as it is WHY you drink and if you are able to NOT drink. For me, I drank to relax. Take away drinking and I was too anxious to unwind. Drinking allowed me to loosen up and enjoy my free time, or so I thought.
At first, I could control it and I only drank weekends in the evenings. Then it progressed to weekends and whenever I felt stressed. Then, I was drinking around lunch time, taking a nap, and then studying, etc. I did cut down for a long time, but it never stuck. I could moderate for a few months and then I was right back at it.
Like you, I struggled with being labeled an alcoholic. I didn't really want to quit. In fact, I still have problems with this sometimes. It just doesn't seem fair! But you know what, it doesn't matter. The fact that it is not fair is meaningless. I have a problem and only I can fix it. I can accept that and be well or fight it and eventually drink myself to death. I chose to accept it.
Your choice is yours alone - we can't tell you what to do or what is right for you. But I think you know the answer already.