I don't think it's what you're doing that's fueling it, it's his loss of control over you. I get how hard it is--I get it as much as someone who hasn't been there can. I've seen what it does to people.
The thing is, as hard as it is now, fighting back is really the ONLY way to make him stop--short of letting this piece of crap control your life until the kids are out on their own. And believe me, he will keep pulling stuff like this, just to prove that he CAN. Making it not be in his own best interest to continue doing this sort of thing is the only way to make it stop.
Do you seriously think that he'll leave you alone if you let this slide? I don't.