Human beings are judgmental by nature. We are all going to be judged by others no matter what. I remember how I used to get looks of pity and/or disgust from cashiers when they'd see me buying beer again and again.
But I generally deal with situations you described on a need to know basis. And everybody in the world doesn't need to know why I don't drink. In fact, pretty much nobody but me. And I'm fine with it!