I'm with you - I do a lot of outdoorsy stuff, and - in the past - many of those activities are defined by drinking. But what I've found is that many of the truly outdoorsy people drink very little...there's a big difference between the folks who drag cases of beer on a car camping adventure and the rugged athletic back-country folk who might include a little alcohol in their backpack for the end of the day, by the campfire. That latter group is not going to try to seduce you into drinking. They tend to be more health conscious, and the experience isn't defined by drinking for them.
So - don't give up camping, but maybe find other people to camp with!! I'm making a big effort to connect with the more committed outdoors folks before summer. I signed up for a kayak trip with people I know are very healthy in their lives, and will support my not drinking. I've found a few folks in AA who are interested in hiking and camping, and we're planning some adventures this summer.
If you do want to go with your drinking buddies, you'll have to consciously BE that other guy - the one who wakes up at 6 am and goes for a trail run, who drinks out of his big water bottle around the fire, the guy who doesn't drink because he doesn't WANT to. Even if you are struggling inside, wear the mask of the guy who chooses not to drink!! This is a far more powerful person to be, and will steady you for any potential conflict regarding your not drinking.
It sounds like these friends don't have your best interests in mind (and it also sounds like they might be very boring camping partners). Consider expanding your social group and finding people who camp, but don't drink heavily (or at all)... There are many!!