Hi tkmal -
Welcome. Glad you are here.
For me, it started like your story, a few drinks a couple of times a week, then a few drinks every night, then daily drinking and a few nights with a lot of drinks... (all while being an athlete, as well)
you get the picture... it is progressive.
So, I finally decided that I didn't want to keep drinking, so I tried to stop. And I couldn't. I'd stop for a few days and then start feeling like "I deserved it" or "It has been a bad day, so a drink would be nice..." all the stuff we tell ourselves when we aren't really being honest with ourselves.
My favorite one was how I would pour myself an 8 oz glass of whiskey and call that 1 drink... but that is another story for another day...
Anyhow, as others have said, only you can decide. The best test that I found was to try to stop drinking for 30 days. If you can do it without craving for alcohol, then you probably just need to cut back to get back to your healthy lifestyle. But, if like myself, you have crossed that invisible line into alcoholism, you won't be able to do it.
If it bothers you, you should stop - and if you aren't an alcoholic, you will be able to. If you can't, then you're in the right place. Be honest with yourself. Keep posting.