You know what's worse than the guilt of leaving? The guilt of staying. Leaving is like a breaking a finger, staying is like breaking a femur. Both hurt, but one hurts a lot more than the other and people often die from it.
The single biggest mistake I have ever made in my 45-Year old life, and I've made plenty (some doozies too), was staying. It's number one with a bullet.
Take what you want and leave the rest.
Cyranoak