You have the right to change your mind. That being said, if you don't want to approach it that way, then clearly state your boundary for the next paycheck-no extra groceries for him.
As for the mediator/counselor, again, I am concerned about just how 'healthy' visitation is for those kids when he can't make that a priority. He's doing what active alcoholics do, which is only thinking of himself.