In case anyone was curious about what had happened I thought I would give an update.
When my father was contacted by the social worker he convinced them he did not need any help. Apparently his words were, "I don't need that ****."
He came out of his room to proudly announce to my adult brother and husband that the visitation won't happen.
So far nothing has come of it.