There is a difference between valid detachment, and turning your back on someone. The former entails compassion, the latter contempt.
I get the sense that, if the kindness in your heart could fix this man, you would use it to fix him, even today. That is compassion. But now you know it cannot fix him, so have detached.
Please give yourself kindness in this awful situation, not judgment.
Said warmly,
CLMI