As hard as this may be to hear, until you have solid sober time in, you may need to take some distance from your family.
When you first do this, they may not understand. When they find out you are not drinking but they still are, they may resent it. You are a trouble maker for not drinking as it makes them think about their drinking. With time hopefully they will accept your choice.
As you get stronger in your sobriety it is possible to resume these types of activities with your family, but you do so knowing it is a risk to your sobriety.