Sounds like it would be best for you to not interact with him at all. Your sister had no business trying to manipulate you with that "he will take it out on me" nonsense.
Personally, I wouldn't go as far as avoiding all family functions he might attend.
I do think you need to move on emotionally. I mean, he's just your BIL. I barely know mine exists, lol.