Yes, I can imagine. Why should they change for him if they don't want to? The option was to not put oneself in situations where this occurred.
Taking back my power has freed me. Setting boundaries allows others the dignity to decide how they will conduct themselves. For example, perhaps deciding to have no contact with the family as long as they drank in front of me, the family would change in order to be with me. Or not. Then I would know what I needed to do.
Blaming others gets me nowhere.