I've never had a DUI, never missed work for drinking, and never gotten myself into serious trouble. My marriage is happy and my friendships are fine. But I don't like the person I become when I drink.
You have a lot of the classic "I never's". Is it really worth it to risk losing everything? As mentioned above...it's not how much or how often...it's what happens when you do. And, age really doesn't have anything to do with it...the progression can be quick, or it can be slow...but, progress it will.
Why not try quitting for about three months, and see if you don't just feel better about every aspect of your life!?! You have said
I've gone for weeks at a time without a drink and I don't get cravings.
so maybe your drinking hasn't gotten to the point where you actually need to detox. If you need extra help in staying away from the alcohol, AA meetings are a great source of support and inspiration.