Mummyto2,
This type of thing is only going to get worse unless find a way to quit drinking. Your children are getting older, they will notice things themselves and their friends are going to hear things and it just grows. As other suggested, let this be that straw that breaks the back of your drinking. It is the only way this type of gossip stops.