We gotta separate the drinking from the activity, if I wanted to drink I'd go to the pub or a liquor store, whereas if I'm going fishing, to an ice hockey game, to the theatre or the cinema, that's the primary focus of the activity.
It's also important to have people that understand and respect your decision to be Sober, and it sounds like you do.
The issue is simply being around drinking and drinking activities, the temptation of it all, and with 3 weeks of Sobriety it's still early days, there will be many more fishing trips to go on in the future, and the priority here is to protect your Sobriety at all costs!!