It's not wimpy or snobby. I avoided it a lot during my first sober year and even now there are times where if I know everyone is going to be drinking, I may decline to go just because I don't feel like dealing with it. And let me clarify the "it" - that would be the people repeating themselves or slurring because they've had a few drinks. It's not fun to watch other people act that way. One of my biggest peeves now is when people say "oh it must be so fun to be sober and watch everyone act stupid and get drunk." No, it's not...not at all.
I let whoever think what they want ab my sobriety or how I chose to deal with it. They will be supportive if they truly understand. My boyfriend quit drinking for almost my entire first year because he knew that it would help me. I'm not saying you need to ask your husband to do that, I'm just saying people should understand if you aren't comfortable around it.