Gracey,
People told me this many times and I never really understood it:
When it's time to leave, you'll know it.
It was hard to be patient and I wasn't really sure I'd ever get to that point. I just had to trust what they were saying and keep on working on me.
Now, I can say with absolute certainty - When it's time to leave, you'll know it.
I wish that I had taken all of that energy I used trying to figure out whether to stay with him or not and used it to work on myself and work my program.
There is no reason to feel guilty about doing what is right for you.
Hugs - L