I know it seems harsh to not invite your sister to the wedding, but sometimes people do things that are so unacceptable that other people no longer want to be around them. And by losing out on things, sometimes they learn that they have to change. Sometimes they don't. But they never learn if no one points it out to them and sets boundaries.
You have to consider the greater good: who do you want to disappoint? Your sister and her husband, who are the instigators of these problems? Or everyone else at the wedding who would be subjected to their outrageous behavior? Someone is going to have to be hurt, there is no way around it, and it's your sister's own fault, unfortunately.
I will let the other posters who have a lot more AlAnon experience than I offer advice on how you can best help your sister.
But I hope that you find peace in your decision about the wedding. The fact is, it's your daughter's wedding and the decision is really up to her.