There's a big difference between "I feel terrible because you're leaving and I wish you wouldn't leave" and "I understand my part in your decision to leave and I am working on x, y and z to change it. I feel terrible about what I've done and I wish I could make it up to you, but I understand that I can't and you are leaving". It sounds like you're mainly hearing the first, not the second.
Of course he feels terrible and sorry and doesn't want you to go - but it's too late now. It sounds like he had seven years worth of chances, and didn't take them. He can't turn back time.
You're doing the right thing.