I don't really know what to say. I swing between thinkig I can and have imagined and knowing what we imagine is never the reality.
Don't waste time on guilt, I don't believe I could stay to the end either - I also know that when I let D go it didn't mean he forgot he was loved, not for a whole decade - I don't believe people ever really forget that. Perhaps without the decision you made then you wouldn't be able to be where you are now, to have gone to see him as you did and I do believe his heart would know you were there despite the limits of his body.
The are people and I don't believe any of us will ever learn to deal with this if we deny that, a part of us knows a person is complete and stays very much human. You are grienving for someone as valid as any other man on this earth, loving him was not wrong, nor was letting him go, nor is being with him now.