I understand the struggle. To me, it always mattered. I even find today that I wonder "remember that time 2 years ago when we were at xyz, were you drunk then?" For me, not knowing and wondering what is "truth" is worse than knowing the truth even if it is BAD. But I believe in the advice that it really shouldn't matter.
Regarding "How do I know," ... With my family, I know the truth. With my friends, I know the truth. They don't lie to me. But I am slowly starting to realize that I may NEVER know the truth when it comes to an alcoholics behavior. It is really so sad.