Not a fan of "it was just a joke" excuse. Passive aggressive behavior.
I would address it with him. No long discussion. "Stop using me just discipline your children". I don't believe I would address what you feel are the possible repercussions in behavior, because that is just opening up a can of subjective worms and an argument. If he says "It was just a joke" I would respond with "Ok, you were joking? I am not. Stop".
In the future should he continue, using your example of what happened, I would walk into the kitchen, tell her you won't be mad at dad, and that she needs to close the fridge.
Certainly no parents are perfect. I had pretty close to perfect parents and can often remember one using the other for discipline in a slightly different manor "I am going to call your father if you don't stop……." Calling Dad at work meant big trouble. We avoided it.
You can try to get him curtail his behavior. Not sure you will succeed, and not sure how it will affect your children in the long run.