I think your fears are warranted, but they can't stand in the way of your sobriety. Alcoholism I have learned is a brutal disease, it destroys families, cost people jobs and ultimately kills. So saying no and hurting some "feelings" is minor in my opinion.