I worked in an office where I was the only smoker.
When they would hound me about why I don't drink (people can be so rude), I would reply, "for the same reasons that you don't smoke." They would try to argue, "but smoking is unhealthy for you and it smells so bad on people, it's disgusting." Well, there ya go.
It's nobody's damn business why you do or don't do something. You don't owe anyone a detailed explanation for your preferences.