So, from the other side, looking on. I have a neighbor who does not drink. I have known that as long as I have known him, or at least since the first time I offered him a drink or invited him to a party. That would have been when I was drinking regularly.
His answer was, or maybe his wife mentioned to me beforehand, "Joe doesn't drink." My next thought? Oh, what does he like to drink, so we can get some for the party? I am sure MOST people just shrug it off much the same. You just figure when someone says it so definitvely, they have a good reason, whatever it is. Health, personal preference, bad history with it. whatever.
I think we fret about it more than they do! and in a new group, all the better. Easier than changing in the group you used to drink with. They will just get it and offer you a diet coke.