If coffee is your trigger to drink then don't drink it.
I'm also a coffee drinker, have been my whole life, but I only have one cup or two in the morning with breakfast.
Coffee itself is not bad, but if that is what is triggering you to do more harm to yourself than it IS bad.
Try changing your routine, or at least the amount of caffeine. Less coffee, more milk, or something like that.