Oh, I've definitely had people not leave it alone when I say "I don't drink".
Me: "Oh, no thanks, I don't drink."
Them: "Never?"
Me: "No, never."
Them: "Like never? Like not even on New Years Eve?!"
Me : "Nope."
Them: "OMG, well what do you do for fun?"
Me: "hahahahaa"
If they are interested I tell them I I have plenty of fun...been skydiving, parasailing, oh and I've run in two naked 5ks with 300 other naked people. When I ask if they'd care to join me on the next Bare Dare, they usually say "OMG I could never do that!" I say, "Really...well, what do you do for fun?" lol I've yet to convince a heavy drinker that drinking is not really considered a hobby.