I said the same thing....YOU did hurt her when you did this, YOU hurt her for that, YOU, YOU, YOU. And of course it put up his defenses and the conversation stood still.
I'm not sure even wording it would be different. With my xabf everything was a personal attack on him, regardless of what I said or how I put it. He had to be the victim, even if whatever was being said wasn't being directed at him.
I know what you're going through, wish I had some thoughts for you. Like someone else says here--no IS an answer. If there's not a legal visitation set up, simply tell him no. The offer stands to come over for an hour or two and then ask if she wants to go but ultimately it's her decision. He can take it or leave it.