I think you handled it very well and didn't need to say any more. The vast majority of people have no problem with people NOT drinking alcohol, and why would they?!
The people who do have a problem.......well we can guess the reason for that. Perhaps we can pray for them to enjoy life sober, like we are learning too.
Well done